research

Artificial Intelligence is the Future of Peer Review

Photo by Steve Johnson on Unsplash.

It is expected that in 2025, approximately three million articles will be indexed in Scopus and the Web of Science. If each undergoes peer review by two experts, and an additional 2 million articles undergo peer review, but are rejected—approximately 10 million peer reviews will be conducted this year—a staggering number that is likely to grow as the biomedical enterprise, and the number of peer-review journals increase.

According to an editorial in the journal Critical Care Medicine, in the coming years, artificial intelligence (AI) should be part of the future of peer review.

“Peer review at biomedical journals has been essentially unchanged for many decades. Although compensating peer reviewers would likely help to receive timely reviews, it is probably not feasible on a wide scale. In addition, peer review has well-known limitations,” said Howard Bauchner, MD, professor of pediatrics. “We believe peer review should include some form of initial review by AI, assisting editors in decisions on which articles to send out for external peer review,” adds Bauchner, who also is former editor-in-chief of the Journal of the American Medical Association.

Bauchner outlines the limitations of peer review and defines the various types: double-blind, single-blind and open review. He describes one of the largest trials ever conducted comparing double-blind to single-blind review. “When reviewers were aware of the authors’ identity (single-blind), they gave a more favorable rating from countries with higher English proficiency and higher income. These findings are consistent with what has been known for years: peer reviewers can be biased. While Bauchner agrees that AI could also be biased, he questions whether it is more biased – than a human peer reviewer. He believes models could be taught to disregard who the authors are and where they come from.

Bauchner also stresses that several independent groups which already offer AI review of articles, largely as a service for authors prior to submission of articles, have already experienced good results. He cites one particular study, where the authors found feedback from GPT-4 review to be more helpful than feedback from some peer reviewers.

Additionally, he believes that AI will be good at evaluating whether an article follows the appropriate reporting guideline, which is often noted by authors as requested by journals, but with no evidence that peer reviewers actually check adherence to these guidelines. Furthermore, Bauchner feels AI may be able to detect fraudulent research more effectively than peer reviewers.

“As it continues to improve,” he said. “It is time to embrace a different approach, an approach that is likely to be more efficient and more effective—review by AI.”