Picture this. The same article receives these three reviewer recommendations:
• Reviewer #1: Recommends rejecting the manuscript for publication. Comments include a need to demonstrate that analyses were two-tailed tests, better defend the use of a certain statistic to judge model fit, and add several points to the Discussion section.
• Reviewer #2: Recommends that the authors revise and resubmit. Comments, which include disparaging language (“surprisingly weak,” “obvious to others in the field”), note cumbersome writing, insufficient literature review, and overly long Discussion section.
• Reviewer #3: Recommends that the editor accept the paper. Comments include suggestions on how to clarify the research questions, details that should be added on the sampling and data collection approach, a recommendation to add a table of correlations, and aspects of the findings that could be highlighted in the Discussion.
As editor of a research journal, if I receive the three hypothetical reviews summarized above, should I focus on the confidential recommendations regarding acceptance or on the review content? Should I rescind one if I find it completely inaccurate or unnecessarily hurtful? What action is best for the journal? What is best for the state of knowledge in the field? What is best for the author?
Editors balance these questions daily as they make final decisions on manuscripts submitted for publication. Their decisions are shaped by the journal and its audience, the nature of the writing talent available, the level of competition for acceptance and page space, and many other factors, but some common principles can be found.
The first principle in my decision-making is that the reviewers’ recommendations are not a vote and I have no obligation to follow the majority, or indeed, any of the recommendations. Instead, I use reviewers as consultants. Reviewers are selected because they have indicated when joining my review panel that they have relevance to the manuscript. Plus, I know the strengths of most reviewers. If a statistician has a comment about statistics, it carries more weight with me than if a clinical leader makes such a comment. Conversely, if a statistician comments on the relevance of content to health care clinician readers, those comments will be taken with a grain of salt.
The second principle is that the editor is the arbiter and interpreter to the author of contradictions and confusion in review content. That does include the option to rescind, edit, or de-emphasize a review. I do sometimes edit adjectives and adverbs in reviews to make them civil, and communicate with reviewers whose approach is consistently unhelpful. If reviews are very discordant or complex, I include in the decision letter some guidance as to how to prioritize. I generally support the well-reasoned, well-defended, and important comments and put less emphasis on others.
In making a decision based on the reviews above, I would probably keep the paper in the pipeline and ask for revision and resubmission. No reviewer identified a fatal flaw in the study being described that cannot be remedied in revision. In the reviews above, #1 and #3 provide concrete issues that can be addressed, but #2 provides few specifics on where the problems lie. Authors who are uncertain how to respond to a mix of feedback can and should contact the editor to gain a better understanding of how to proceed.
The peer review process is essential to progress in the literature in any field. If you are a reviewer, please consider yourself one of several important judges of, and contributors to, the quality of presentation of a future contribution to the field, rather than the sole decider of its fate. Whether or not you think the paper has a future in print, provide concrete and polite suggestions without overplaying your own research, and trust that the editor does not expect you to comment on aspects of the paper in which you are not an expert. If you are an author of a work in revision, please use your editor for guidance as needed to respond to conflicting recommendations. If you are an editor, take the time to scan each review for plausibility and civility, and assist the author as needed to separate the wheat from the chaff and give each review the attention it merits.
Credit Image/Source - iStockphoto