Peer review is at the core of contemporary science. Although it has been doing an excellent job keeping junk research out, it has many flaws. I have a few ideas that could be used to improve the system.
Academic science turns around papers. As scientist, you do your research and then you make it public in a specialized journal. But before your paper is published, the journal editor will ask one or more specialists in the field of your research to review your paper and tell whether your findings are worth being communicated or not. This is the peer-review system and it is the core of science today.
Along the years, I’ve been playing the three roles: reviewer, editor, and specially author. Based on this experience, I collected a few ideas that could be tested to improve the system.
1. Clarification step
If someone tells you something and you do not understand well, what do you do? You say something like, “Pardon me, could you please explain your question?”
Reviewers can be awfully obscure sometimes. Authors often have to guess what they want, without the chance of asking for clarification.
The first step in the review process should be to grant the authors a period – say 48 hours – to request clarifications about the reviewers’ reports.
2. Review check-list
Reviewers’ reports are completely arbitrary. Apart of few general journal guidelines, each reviewer will focus on what he or she wants.
Editors should take reviewers by hand and guide them through the revision. A check-list is the way of doing that:
- Is the abstract consistent with the paper findings?
- Does the introduction adequately survey the literature? Anything missing?
Many top journals are adopting a “two positives or die” practice.
Editors send the paper to two reviewers and it is considered for publication only if it gets positive recommendation from both. This is bad. The reports are very much subjective. It’s comically common to get:
- Reviewer 1: “Awesome work! Publish as is.”
- Reviewer 2: “Are the authors out of their minds? Throw it into the garbage can.”
In such tie situation, to reject is a non-sense. But to save time and work, editors do it!
Please, editors, just ask for a tiebreak report.
4. Pay for review
Review work is unpaid. It is a tradition in the scientific community to do that for free. But we know, nothing is for free.
Reviewing a paper costs several working hours. Who is paying for that time? Of course, either the institution where the reviewer works or the reviewer, if the work is done in the free time. Thus, scientific journals have been cleverly transferring part of their costs to a third-party, many times, the public sector.
The practice of asking for a report on a paper comes from simpler times, where the editor was asking to a colleague a favor. Today, however, science and scientific journals are professional enterprises. They should adopt better practices.
The moral is:
Don’t ask me to do for free, one of the few things I know how to do.
5. Public reviewers list
To be a reviewer for a journal has symbolic value. It shows that the invited person is considered a peer, a specialist in the field. Specially for young researchers, this is an accomplishment that should appear in their CV’s.
However, how can you prove that you reviewed for a specific journal?
Editors could make public a simple general list of all people who reviewed papers for that journal. Simple like that, the value of being a peer reviewer would be acknowledged.
6. Authors shouldn’t do editor’s job: suggesting reviewers
Every journal asks for suggestions of potential reviewers (and also of people who should not review).
In principle, this is not bad, but it is often annoying. You have to waste time trying to remember people who could do the job well. Then, you tend to suggest big-shot names to show that you are not afraid of the revision, but also knowing that big shots are not going to take papers to review.
Editors could help with that by providing lists of potential reviewers from where authors could select.
. . .
That’s enough for today. I will tell the remaining 5 ideas to improve peer review in the next post.
- What do you think? Would these ideas help to improve peer review? Did I miss anything? Leave your comments or like below.
- If you enjoyed this post, follow the blog by signing up in the sidebar. You can also follow me on Twitter.
- Do you know how much $papers a scientist is worth? Check this post.
- The illustration is based on the painting “The Alchemist” by David Teniers the Younger.