Thursday, 10 October 2013 00:00

Open-Access—The Bad and The Good

Written by 
Rate this item
(2 votes)

Open Access logoGrace Groovy of the International Journal of Cancer and Tumor would be happy to publish your nonsensical data, Science Magazine news reporter John Bohannon discovered. But that journal wasn't the exception. In a 15-month investigation, Bohannon concocted a bogus paper that he fed into a program that randomly generated bogus variations from bogus researchers at various bogus institutions which he then submitted to 304 open-access journals. In all its forms, the paper lacked even basic experimental controls and yet out of the 255 journals that responded, 157 accepted the paper for publication. Grace Groovy's acceptance was only too typical of the non-existent peer-review by the journals that accepted it.

Bohannon's story is a damning indictment of scam artists who use the open-access model to extract "page fees," particularly from scientific authors in the developing world desperate to see their work published anywhere. But toward the end of the story, he quotes David Roos, an ASCB member at Princeton, whose complaint about deceptive journals triggered Bohannon's investigation. Roos believes that the open-access model itself is not to blame for the poor quality control revealed by Science's investigation. Bohannon writes, "If I had targeted traditional, subscription-based journals, Roos told me, 'I strongly suspect you would get the same result.' Bohannon, however, concludes that "open-access has multiplied that underclass of journals, and the number of papers they publish."

University of California, Berkeley scientist Michael Eisen agrees with Roos and points out that subscription-based journals publish bad papers too. In his blog, he gives the example of a paper published in Science with all the flaws of Bohannon's. According to Eisen, the reason any journal accepts a bad paper is due to the conventional peer-review process. Eisen wrote in his blog; "If we had... a system where the review process was transparent and persisted for the useful life of a work... none of the flaws exposed in Bohannon's piece would matter."

Eisen is a co-founder of the Public Library of Science (PLOS), a nonprofit open-access publisher. Ironically, all PLOS publications use a traditional peer-review process. Just like in subscription-based journals, PLOS editors decide if the paper is acceptable for the journal; if so, it is sent to be reviewed by multiple anonymous academics.

At seven years of age, PLOS ONE is the grand old man of "easy" peer-review with review criteria focused on determining whether conclusions are presented appropriately and are supported by the data, rather than evaluating the significance or novelty of the work. When PLOS ONE first started, there was a widespread perception that getting accepted was overly easy. Now PLOS ONE is well respected, with its papers frequently highlighted in the media. In Bohannon's "sting" operation, the editors rejected his bogus paper in just two weeks, citing its inadequate scientific quality.

PLOS ONE's traditional review process did work to weed out Bohannon's shoddy paper. But there are open-access journals that are experimenting with different approaches to review systems that might fulfill Eisen's hopes for peer-review. Some of these journals are described below.

eLife asks peer-reviewers to limit recommendations to those that directly impact the conclusions of the paper. If a reviewer believes the paper requires additional experiments, peer-reviewers meet online to discuss a submitted paper, and together they decide appropriate feedback for the authors. The reviewers are anonymous to the authors.

F1000Research features post-publication peer-review. After a paper is submitted, it is published online without delay. The authors identify potential peer-reviewers, and after F1000Research confirms that there is no conflict of interest, they are invited to review the paper. All of the reviews are posted online without anonymity. In addition to providing comments, the reviewers select one of three statuses: "Approved" (similar to accepted or minor revisions), "Approved with Reservations" (similar to major revisions) or "Not Approved" (rejected). Then the authors have a chance to edit their manuscript, and reviewers can review their changes. Once a paper has at least two "approved" statuses, the paper is indexed in PubMed, Google Scholar, and more.

PeerJ encourages, though doesn't require, reviewers to reveal their identity. PeerJ is also unique in that to publish, authors must buy lifetime memberships for $99-$299 each (depending on how frequently the author plans to publish). Once authors become PeerJ members, they can publish subsequent papers without charge, provided they contribute to the journal by commenting or participating as a reviewer at least once per year. PeerJ also offers PeerJ PrePrints, which allows authors to get feedback before their manuscript is complete.

To compliment the peer-review process, the website PubPeer aims to provide post-publication peer discussion on journal websites. Anyone who wants to make a comment or ask a question about any paper listed on PubMed can do so anonymously (although academics who register will have their comments labeled as "peer" and outsiders will be "unregistered submission"). Then the authors will be notified of the comment and can choose to respond. Back at Science, Bohannon is unlikely to repeat his journalistic experiment and test all 9,641 open-access journals. The question thus becomes whether experimental approaches to mass peer-review such as PubPeer can spot bogus papers and irresponsible journals.

Christina Szalinski

Christina is a science writer for the American Society for Cell Biology. She earned her Ph.D. in Cell Biology and Molecular Physiology at the University of Pittsburgh.

Email This email address is being protected from spambots. You need JavaScript enabled to view it.