Reforming the peer-review process
Slate has an excellent (albeit old) article critiquing the peer-review process currently in place for selecting research articles for publication in medical journals and other scientific proceedings.
In the article, they bring up the suggestion of “pre-publishing” studies online and essentially allowing a less-structured and informal peer-review process. Those studies deemed worthy (although it’s not clear exactly how this would work or what criteria need to be met) would go on to formal publication. I like this idea and think it addresses many problems with our current peer-review system.
Fundamentally, the problem with peer-review is that it is a system that was designed when less research was conducted, studies were less complicated, and formal communication of scientific ideas was limited to printed publications. In this context, it makes sense to have a limited number of experts in their field reviewing research and judging its worthiness for inclusion in the formal literature. The limited space afforded by physically printing journals also lends itself to the basic structure of research articles (i.e.–background, methods, results, conclusions). However, given the explosion of scientific research and the advent of various forms of electronic communication, this system does not make sense anymore.
A wider array of experts in all fields now exists, much more so than 50 years ago. This deepens the pool of potential reviewers, yet current peer-review processes for most medical journals rely on a small, select group of senior researchers from a given field. This results in a positive feedback loop and tends to create tunnel vision in the review process. The narrow group of editors and reviewers have their own research interests/expertise (which they published on extensively to get their “reviewer” status) and consequently enjoy seeing research in those areas. (Not that these people have devious intentions; this functions on an unconscious level…most of the time.) Having a wider, more diverse group of reviewers would result in a broader range of published articles and possibly lead to more diverse suggestions to authors during the review process, ultimately producing more forward-thinking publications. This process is facilitated by the ease of electronic communication and online tools.
More importantly, medical journals need to capitalize on the benefits of electronic publication. The ability to publish materials online has enormous potential for improving the scientific literature. As suggested in the Slate article, journals could create online sections for research that may not be ready for primetime. Members (since many journals are publications of professional societies, they could cut out some of the riffraff by limiting comments to society members) could then critique studies, provide valuable feedback, offer to collaborate, and ultimately advance mediocre studies to full publication. The ability to publish online also means journals can include more than the basic background, methods, results, and conclusion typical of almost all scientific publications. Some journals have already done this by allowing authors to include additional tables and figures or copies of survey materials online. This needs to become the norm rather than the exception for publications. Ultimately, researchers should provide primary data that can be re-analyzed and independently cross-checked to ensure accurate and appropriate analyses were conducted. Such levels of scrutiny would represent true peer-review.
Although ongoing efforts to assess our current peer-review process are valuable, I think it is more important to capitalize on recent technological advances and move forward with some of the more obvious changes to how we publish medical research.