Academics accuse AI startups of co-opting peer review for publicity

AI startups face backlash for using peer review to promote AI-generated studies.

: Sakana, Intology, and Autoscience claimed to have used AI to generate studies accepted for ICLR workshops, leading to criticism from academics. Sakana gained consent from reviewers but Intology and Autoscience didn't, sparking concerns over the misuse of the peer review process. Prithviraj Ammanabrolu and Ashwinee Panda voiced disapproval, citing disrespect for reviewers' time. The controversy highlights the growing issue of AI-generated content in academia and potential need for regulated evaluation.

The controversy surrounding AI-generated research papers submitted to a key AI conference, the International Conference on Learning Representations (ICLR), illustrates a significant debate in the intersection of AI development and academic ethics. Three AI laboratories—Sakana, Intology, and Autoscience—reported that they had used AI to generate studies accepted to ICLR workshops. The critique primarily revolved around Intology and Autoscience's failure to forewarn or obtain consent from conference organizers before submission, a departure from the approach taken by Sakana, which informed ICLR leaders and obtained consent from peer reviewers for its 'experiment' before submitting the AI-generated papers.

Prominent voices emerged in the academic community to decry the actions of these two labs. Prithviraj Ammanabrolu, an assistant computer science professor at UC San Diego, chastised the involved AI companies, expressing on social media platform X that the manipulation of peer-reviewed venues as human evaluation channels without consent undermines the dignity of the scientific process. Such criticism was echoed by Ashwinee Panda, a postdoctoral fellow at the University of Maryland, who posted similar sentiments on X, outlining the lack of respect exemplified by these submissions for human evaluators' time and effort.

A critical concern underscored by these critiques is the resource-intensive nature of the peer review process. A Nature survey cited by commentators highlighted that around 40% of academics spend two to four hours reviewing a single study, a task undertaken predominantly as volunteer work. The workload is intensifying; submissions to NeurIPS, the largest AI conference, increased 41% to 17,491 in the previous year alone. The infusion of AI-generated papers into this overburdened system further exacerbates concerns around workload and resource allocation.

Additionally, the prevalence of AI-generated content is a pressing issue within academia. Reports indicate that between 6.5% and 16.9% of submissions to AI conferences in 2023 possibly contained synthetic text, posing challenges to the integrity of scientific dissemination and study validation. Sakana's experience highlights these challenges, admitting 'embarrassing' citation errors within their AI's output and later retracting their submission citing transparency and respect for conference conventions as key reasons.

To resolve these emergent issues, suggestions have been made by figures like Alexander Doria, co-founder of AI startup Pleias, who advocated for an organized, regulated approach to AI-generated study evaluations. He called for the establishment of a dedicated agency to perform high-quality assessments of AI-produced research, a process that should include compensating researchers for their contributions, as academia should not serve as an avenue for unsolicited, pro bono evaluations of AI technologies.

Sources: TechCrunch, Nature

AIResearchethics