Not long ago, I asked a top-notch scientist how often he accepted papers to review. His answer was curious. He told me: “I publish 30 to 40 papers a year. Each one takes two or three reviewers, making about 100 reports. I feel that I have to give this same quantity of reports back to keep the system working. Therefore, I review about 100 papers a year.”
I was a bit surprised. First by the reasoning, which was new to me; second by the number. I asked him how could he find time to review 100 reviews per year, roughly two papers per week. And I was again surprised by his answer: “I don’t really do it myself. I ask my students to review them.”
I think he read the ethical concerns on my face, as he continued somewhat defensively: “I think that this is a good exercise for my students. They learn how to critically read a paper, survey references, write a technical report. Of course, I check each review before submitting it.”
I don’t know numbers for that, but I don’t think that outsourcing a review to students is rare. Many of my colleagues told me that, as students, they were asked by their advisors to review papers. The practice is even captured in this PhD Comics cartoon.
When I asked that professor about his “review policy,” I was trying to get some insight to myself. I usually get one, maybe two review requests per week. Naturally, I have to decline most of them, otherwise I wouldn’t do anything but peer-reviewing, which would ironically grant me to be excluded from the “peer” club in no time.
I basically only accept to review papers that are of immediate interest for me. It may sound opportunistic (and it really is), but seeing what other people in the field are doing ahead publication gives me some competitive advantage.
On the other hand, I don’t ever ask students and co-authors to do the job for me. What I often do is to write back to the editor declining the invitation and suggesting that the review request is sent instead to one of my co-authors, who I think is qualified to do the job. Maybe, in few occasions, I also suggested that some especially boring and troublesome papers were sent to people who I didn’t have in good esteem. (Don’t look at me like that. Nobody can be nice 100% of time…)
To transfer the review job to students is against the rules of most journals. Normally, they explicitly state that the person invited to review must not show or discuss the paper with anyone else, and if an additional opinion is required, the editor should be informed.
Even more serious, to transfer the review to students most probably lowers the review quality. I’ve received many reports in the past, filled of primary statements, which clearly denounced that someone with little experience wrote them.
Especially concerning postdocs, who are usually qualified to review a paper, it’s much better to suggest them as alternative reviewers than to ask them to informally do the job. This will help them to improve their CVs and also help editors to be aware of new people coming to the field.
In any case, I liked that professor’s rational that to keep the system working we should give back the same amount of reports as we get to our own papers. Indeed, I’ll adopt it to guide my peer-review policy from now on.
Putting everything together, I’m thinking about sticking to the following rules:
1. I will not ask any other person to do the review for me.
2. If I should decline an editor’s invitation, I may suggest one of my young co-workers holding at a doctor degree as an alternative reviewer.
3. In a given year, I will review a number of papers that equals the number of reports I had in the previous year divided by the number of my co-authors holding a doctor degree. <This rule is bullshit, see the Erratum below.>
As for the last rule, let me give an example. In 2014, I published 10 peer-reviewed papers. It was a good year, there were no rejections or resubmissions. These papers—I just counted—demanded 25 reports and were co-authored by 24 colleagues holding a doctorate. Therefore, each of us should give back one report now in 2015 to break the system even.
These rules sound fair, but I can’t avoid thinking I’m missing something about the last one <And in fact I was…>. If I’m supposed to review only a single paper per year to keep the system balance, why do I still receive over 50 invitations to review every year? Are the editors too restrictive with their reviewers’ lists? Do too many people suggest me as a potential reviewer when submitting their papers?
Anyway, I should have thought of that third rule before. Only one or two reports per year, I do really like how it sounds.
It also means I already did too much review work this year. And, you know, I hate overworking.
- More about reviewing papers? Check 11 Ideas to Improve Peer Review.
- Subscribe in the sidebar to know about new posts.
- The painting opening this post is the Nicolaes Maes’ An Old Woman Dozing over a Book, 1655.
Erratum or “Ops, my bad…”
In the comments below, Richard Mostert pointed out that my Rule 3 was fishy. He wrote:
“Your reasoning is sound if you wrote the 10 papers as a group of 25 researchers. Then one each on the basis of these ten papers evens things out. When one of these papers was a review article with you as sole author, you ‘owe’ the community 2.5 reviews just for publishing that single paper. Your 24 co-authors on the other nine papers will not do that for you, in all fairness.”
And he’s right.
Using the example I gave, my publications in 2014, I should count how many reviews I should do to even things out, in the following way:
For Paper 2, with 3 reports and 9 PhD authors, each author owes only 1/3 of report. But for Paper 3, where I was a single author, I owe 3 full reports. Summing over all papers, I should review 10 papers in 2015 to break things even.
Then, I should re-state rule 3 as:
3. In a given year, I will review a number of papers that equals the number of reports I individually received in the previous year. This number is given by the reports/PhD-authors ratio for each paper published in the previous year, summed over all those papers.
It’s amazing how we can always use mathematics to get an answer reasonable, logical, and wrong.