Skip to content

Reforming the peer-review process

December 16, 2010

Slate has an excellent (albeit old) article critiquing the peer-review process currently in place for selecting research articles for publication in medical journals and other scientific proceedings.

In the article, they bring up the suggestion of “pre-publishing” studies online and essentially allowing a less-structured and informal peer-review process.  Those studies deemed worthy (although it’s not clear exactly how this would work or what criteria need to be met) would go on to formal publication.  I like this idea and think it addresses many problems with our current peer-review system.

Fundamentally, the problem with peer-review is that it is a system that was designed when less research was conducted, studies were less complicated, and formal communication of scientific ideas was limited to printed publications.  In this context, it makes sense to have a limited number of experts in their field reviewing research and judging its worthiness for inclusion in the formal literature.  The limited space afforded by physically printing journals also lends itself to the basic structure of research articles (i.e.–background, methods, results, conclusions).  However, given the explosion of scientific research and the advent of various forms of electronic communication, this system does not make sense anymore.

A wider array of experts in all fields now exists, much more so than 50 years ago.  This deepens the pool of potential reviewers, yet current peer-review processes for most medical journals rely on a small, select group of senior researchers from a given field.  This results in a positive feedback loop and tends to create tunnel vision in the review process.  The narrow group of editors and reviewers have their own research interests/expertise (which they published on extensively to get their “reviewer” status) and consequently enjoy seeing research in those areas.  (Not that these people have devious intentions; this functions on an unconscious level…most of the time.)  Having a wider, more diverse group of reviewers would result in a broader range of published articles and possibly lead to more diverse suggestions to authors during the review process, ultimately producing more forward-thinking publications.  This process is facilitated by the ease of electronic communication and online tools.

More importantly, medical journals need to capitalize on the benefits of electronic publication.  The ability to publish materials online has enormous potential for improving the scientific literature.  As suggested in the Slate article, journals could create online sections for research that may not be ready for primetime.  Members (since many journals are publications of professional societies, they could cut out some of the riffraff by limiting comments to society members) could then critique studies, provide valuable feedback, offer to collaborate, and ultimately advance mediocre studies to full publication.  The ability to publish online also means journals can include more than the basic background, methods, results, and conclusion typical of almost all scientific publications.  Some journals have already done this by allowing authors to include additional tables and figures or copies of survey materials online.  This needs to become the norm rather than the exception for publications.  Ultimately, researchers should provide primary data that can be re-analyzed and independently cross-checked to ensure accurate and appropriate analyses were conducted.  Such levels of scrutiny would represent true peer-review.

Although ongoing efforts to assess our current peer-review process are valuable, I think it is more important to capitalize on recent technological advances and move forward with some of the more obvious changes to how we publish medical research.

Advertisements
One Comment leave one →
  1. January 2, 2011 3:37 am

    Excellent point, thanks for the tipoff. This seems to be part a long historical story about what science is or should be; there must be some structure of authority for the process to achieve something but not so much that it stifles creativity and new ideas. As you point out though, there is no urgent need to decide what sort of peer review should fit into the structure, what is crying out to be improved is the use of technology at hand. I completely agree with your suggestions and I’ve made similar comments on other blogs.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: