This is part 2 of a blog written last month. In Part 1, I explained how open access (OA) journals work and some of the shortcomings of peer review. This was done to provide a background on a recently published study by staff at the leading journal Science. In this part, I will cover the specific experiment reported by Science and explain some of the limits of its design followed by an interesting and novel model of the non-profit OA journal BioéthiqueOnline.
Part 2: Open Access Journals, Peer Review, and Conflicts of Interest
Do OA Journals Perform Rigorous Peer Review?
Recently, John Bohannon of the Science group conducted an investigation where he submitted scientifically flawed papers using fake names and provided the names of research or academic institutions that didn’t exist to 304 OA journals (Science 342: 60-65, 2013). The idea was to create a scientific paper with major errors, so that “[a]ny reviewer with more than a high-school knowledge of chemistry and the ability to understand a basic data plot should have spotted the paper’s shortcomings immediately.” Bohannon created a database of molecules, lichens and cancer cell lines and ran them through a computer program to generate unique papers, but with a standard structure: “molecule X from lichen species Y inhibits the growth of cancer cell Z.” He also created fake authors from fictitious African institutions with the hope that using developing world authors would lessen suspicion by journal editors. The main flawed graph showed a dose-dependent decrease in cell growth yet despite rising concentrations, the effects on cells were modest. In addition, the anti-proliferative molecule was dissolved in a large amount of ethanol and because the control group was not treated with the same solution buffer, the anti-proliferative effects seen could simply be due to the cytotoxic effects of ethanol. In a second experiment, Bohannon attempted to show an “interactive effect” by adding the molecule with radiation, but the control cells were not exposed to any radiation. As the experiments had a tragically flawed design, the idea was that any peer reviewer should pick them up and the article should be rejected.
The results of the investigation were interesting. While some OA journals that Bohannon examined seemed not to exist anymore and others took too long to review before the analysis was completed, 255 submissions did come back: 157 OA journals accepted the paper while 98 rejected it. These results show that over 61% of journals accepted the flawed journal article. Of the 255 submissions, 60% of journals made a final decision without any signs of peer review. Of the 106 journals that did perform a review, 70% accepted the paper. These results are certainly shocking and raise serious questions about the peer review process of some OA journals. Yet, there were several journals that did reject the article, reporting problems with the experimental design. But this study of Bohannon has its own fair share of ethical and scientific problems.
First, the research reported by the Science article has a less than ideal design; there was no comparator arm. So how do we know that the peer review of non-OA journals was any better than what was observed with OA journals? Perhaps the flawed and fake article might have also passed muster in the peer review process of many non-OA journals.
Second, there are problems of research ethics. While there are some ethical issues in the use of deception in research, it isn’t prima facie unethical to use deception in research. Quite a bit of behavioral research uses deceptive methods as necessary to be able to better understand the behavior being investigated. Science had to deceive OA journal editors into thinking that this was a legitimate article and went to substantial lengths to create fake names, institutions, molecules, lichen species, and cancer cell lines. While I have no problem with this research using deception, there are a few ethical quandaries that need to be questioned. While this experiment might fall into the realm of investigative journalism, it looks and smells a lot like research. If it is considered research, the next logical question is whether it needs to receive some form of ethics review. Certainly journalistic investigations fall outside the scope of public health service-funded research in the U.S. and thus would not require Institutional Review Board (IRB) review and approval. But it is also difficult to say whether the research involves human participants. Certainly if the research involves capturing the perspectives of journal editors or peer reviewers, it would be considered human research and require IRB review. Yet there are quotes from editors that serve to give the article that journalistic flare and I remain unclear as whether this falls into the realm of journalism or research involving human subjects. Regardless, my colleague here at AMBI also interested in this study did cleverly mention that whether it is journalism or research, and whether it is research involving humans or not, being that it came from Science — one of the highest profile scientific journals in the world published by the American Association for the Advancement of Science — it might have been good practice to have some ethics oversight and scientifically rigorous controls. At minimum, it would’ve been nice to see the author address this research ethics issue a bit within the article itself.
Third, there is a possible story surrounding conflict of interest. The journal Science is not a full OA journal similar to the ones investigated as they charge for the subscription of articles to their readers. So do the editors of Science not have a vested interest in claiming that their competitor OA journals perform less than adequate peer review? While I understand that the data presented speaks for itself, Bohannon might have decided to declare this conflict of interest. At a minimum, we routinely ask researchers from companies or who are paid by private sources of funding to declare potential or real conflicts in papers permitting the reader to judge the merits of the research. I think it would have been appropriate for a similar sort of declaration to be made in the article.
So while I can point to a few inadequacies of the current paper by Bohannon, I must say that the results are very interesting and frightening at the same time. Despite these drawbacks, the paper does show that there may be a problem with peer review by OA journals, perhaps due to the conflicting interest of making money from authors who are successful at publishing in their venue (see Beall’s List of predatory scholarly open-access publishers: http://scholarlyoa.com/publishers/). Yet, I think there may be one way to remove the conflict, that is, to have OA journals remain not-for-profit.
A New Breed of Open Access Journals: BioéthiqueOnline
For the past couple of years, I have been on the editorial board of a relatively new, fully OA journal called BioéthiqueOnline (BO). BO is a bilingual (French and English), international OA journal which doesn’t charge authors to publish or readers to read articles (http://bioethiqueonline.ca/en/). A collaboration between the Bioethics Program and the Bioethics Student Association at the University of Montreal, in Montreal, Canada, this journal is run primarily by graduate students. In this sense, it is similar to some of the traditional law journals that come out of American universities and colleges. The aim of the journal is to foster dialogue and report conceptual and empirical scholarship in bioethics. The journal publishes a range of article types (e.g., peer-reviewed articles, commentaries, reviews, editorials, art work, and case studies) on a range of bioethics topics and jurisdictional contexts. The journal has transparent and clear guidelines on publication ethics, the peer review process, conflicts of interest, and editors’ roles and responsibilities. BO is also a member of the Committee on Publication Ethics (http://publicationethics.org/).
BO began as a strictly editorial-reviewed journal publisher, but then quickly switched to a peer review process (http://bioethiqueonline.ca/en/archives/2904). The peer review process is done in the same way as for other peer reviewed journals, but with the addition of up-front editorial support for authors needing help to get a manuscript ready to go out for peer-review. A unique thing about BO is that it doesn’t charge researchers to publish. Although there were initial costs associated with starting up the journal, and ongoing maintenance costs, all the editors are volunteers (much like editors for other OA and non-OA journals) and the journal does not have any paid staff. The volume of submissions might not – for the moment – be as high as other journals, so the journal is still able to run in a completely not-for-profit manner. Nonetheless, some sort of stable funding is necessary to ensure journal operations, so the Executive editors are exploring public funding options to support and enhance the journal’s activities. This is important, because one way of handling situations of conflicts of interest is to simply remove the competing interest. While I am not advocating that all OA journals should alter their business model, certainly there is merit and value in managing a not-for-profit journal so that the secondary interest in making profit is removed. This is also not to say that all for-profit OA journals have poor reviews because they are for-profit. Certainly, I have reviewed for some OA journals and published in a few, and the reviews I received were quite thorough and the editorial decisions I found were appropriate for my research and for the research I reviewed.
I think OA is a great model in that it attempts to provide fair access to university scholarship. Certainly, the Bohannon study highlighted the issue of poor peer review, and others have discussed the predatory nature of many OA journals. While there are problems with some OA journals, I think it is unfair to clump all of them together or to dismiss the model because it has issues, which to be fair, may be growing pains in this innovative publishing approach. As already mentioned, peer review of non-OA journals has had its fair share of issues. But the OA model is still young and the bugs can be worked out. I think one model that deserves attention is the BioéthiqueOnline OA model where the journal is volunteer-run and there are no financial interests at stake; at the same time, finding basic funding to operate and support quality publication is an ongoing challenge. Having oversight over editorial decisions and possibly removing the financial incentives might be two avenues to enhance peer review, increase transparency and promote accountability, and so ensure that the OA model being tested by the international academic community succeeds in it aspirations of increasing accessibility to research findings and scholarship.
I would like to thank my colleague Dr. Bryn Williams-Jones at the University of Montreal for his thoughtful comments to this blog.
The Alden March Bioethics Institute offers a Master of Science in Bioethics, a Doctorate of Professional Studies in Bioethics, and Graduate Certificates in Clinical Ethics and Clinical Ethics Consultation. For more information on AMBI's online graduate programs, please visit our website.
|Tweet||0 comments | Topics: AMBI News , Conflict of Interest , Publication Ethics , Research Integrity|
SEARCH BIOETHICS TODAY
SUBSCRIBE TO BIOETHICS TODAY
ABOUT BIOETHICS TODAY
BIOETHICS TODAY is the blog of the Alden March Bioethics Institute, presenting topical and timely commentary on issues, trends, and breaking news in the broad arena of bioethics. BIOETHICS TODAY presents interviews, opinion pieces, and ongoing articles on health care policy, end-of-life decision making, emerging issues in genetics and genomics, procreative liberty and reproductive health, ethics in clinical trials, medicine and the media, distributive justice and health care delivery in developing nations, and the intersection of environmental conservation and bioethics.