Editors: The Unsung Heroes of Peer Review?

September 23, 2016 Verity Warne

86462101.jpgMuch of the discussion around peer review focuses on the role and responsibilities of peer reviewers. But the role of the editor is often overlooked in our current discourse. While reviewers make evaluations of the quality and impact of a research paper, it’s the editor who then uses this assessment to reach a final decision about a paper, and mediate between author and reviewer.

In this post, representatives from Wiley and Publons discuss the role of editors, and how this role may be better acknowledged.

Participants: Samantha Foskett (In House Editorial, Wiley), Daniel Johnston (Publons), Miriam Maus (Editorial Management, Wiley)

Why and how are handling editors important to the integrity of the peer review process?

Miriam: Editors represent a hugely influential stakeholder group in the scientific discourse. Every day, editors make judgement calls that have a direct impact on published research: they select reviewers, they assess the validity of a reviewer report and hold authors accountable to the standards set by the journal they edit. In that way, editors also have a direct impact on an individual researcher’s careers.

Daniel: Absolutely. A quick way to understand the importance of handling editors is to imagine the impact a bad handling editor can have: taking an eternity to commission unhelpful reviews from unsuitable reviewers that demand endless out-of-scope revisions, or lazily recommending publication for flawed manuscripts that hinder the progress of their field, or ignoring warning flags of academic misconduct.

Sam: Yes, but increasingly, it’s not just the quality of the evaluation that counts, it’s how long it takes. And this is where editors have a lot of influence on the author experience. Authors in the physical sciences typically rate speed to publication as one of the top two factors in selecting a journal (along with impact factor). And yet getting manuscripts reviewed thoroughly and quickly by experts is no easy feat. It takes perseverance and access to a wide network.

Miriam: Over the past 5-10 years, the peer review process has undergone a huge amount of change. There was a time when diversity in peer review models pretty much amounted to a distinction between single and double blind review. Add to this open peer review, post and pre-publication review, transferable reviews, reviewing for novelty vs reviewing for scientific soundness and you are only scratching the surface of what is available now. Any of these models comes with benefits and challenges and communities rely on editors to implement approaches that are suitable to their needs.

Sam: More could be done to acknowledge the vital role that editors play here. For me this is about editors getting out in the scientific community and talking about the publishing process and the issues they face – and also, finding out what the community needs. There might be greater acknowledgement of editors if they lead debates on peer review models, for example.

Miriam: As Publishers we facilitate and support new models – through investment in technology and other resources, for example – but we rely very much on the experience of our Editors to let us know what makes most sense for a given journal and community.

With diversity and change comes complexity and editors are often at the crux of challenging issues. Publication Ethics is a big theme and alongside high profile cases of misconduct, fabricated research and fake reviewers, editors deal with such issues in their day-to-day work always aiming – and often succeeding – in identifying issues early on and dealing with them before publication. A recent survey of editors we undertook at Wiley revealed that 82% of respondents see the safeguarding of publication ethics as a primary responsibility of the Editor in Chief role. As a consequence, editors often find themselves at the center of debates challenging established practices and notions of peer review.

Sam: Yes. We find that the resources provided by COPE (Committee on Publication Ethics) are invaluable. Resolving authorship disputes and plagiarism allegations is a somewhat thankless but important task. Few people realize how much time and effort is involved: I can think of one difficult authorship dispute that lasted several years!

Do you feel that the importance of this role is sufficiently acknowledged in the current debate around peer review? If not, why not?

Sam: Editing a respected journal carries kudos and tends to enhance an individual’s standing in the field. So in this sense, there is acknowledgement. But in my experience, the debate around peer review tends to focus more on the role of reviewers. Reviewing is vital of course – but without an editor to assess the validity of comments and to resolve conflicting recommendations, there is no clear outcome for the author. Perhaps more attention is paid to reviewers because established researchers tend to be swamped with requests to review, whereas few people have experience of editing. There can be a lack of awareness of the publishing process.

Miriam: I don’t think this question has a clear yes or no answer. On the one hand, editing or being on the board of a well regarded journal, often has a positive impact on an individual’s standing. In Wiley’ survey of Editors-in-Chief, 83% of respondents said that their reputation within the community was enhanced by being an editor and 44% reported that the role has had a positive impact on their career trajectory. On the other hand, editing work is less explicitly linked to academic progression than authoring published research.

Daniel: It is a great step that we are discussing this for peer review week! Being listed as an editorial board member on the journal’s website does little to convey the nature or the level of their editorial contributions. For some, being on an editorial board is simply a soft commitment to accept a handful of review requests a year.  For others, being on an editorial board means finding and managing 100+ peer reviewers a year.

We can’t forget that academic editors have their own research priorities, and all the requirements to prove their worth to employers and funders that go along with it. Just like with recognizing peer review, the greater the recognition we can provide to editors the stronger the incentives are to put aside their own research for a few hours to do a good job of their editorial assignments.

How might we better acknowledge and/or support editors?

Miriam: Looking at the responses Wiley collected from editors of the journals we publish, it is clear that editors want to be associated with a high quality publication, which provides excellent service to  authors and attracts high quality submissions. Increasing impact factor and download figures are seen as key metrics for success and editors are always looking for information and guidance on how to improve those key metrics. To be able to meaningfully recognize the contributions of editors we need readily available and reliable data that shows how an individual publishing decisions plays out in the medium and long term: Are articles accepted by an individual editor well cited, downloaded, do they have ‘impact’ in the community and beyond? Are rejected articles published elsewhere and how do they fare? Obtaining any quantitative or qualitative data in this way is currently not straightforward, though it should not be beyond the reach of the various players in the research and publishing field to work together to address this challenge.

Sam: Miriam raises an interesting idea about editor metrics. On one hand, it’s an alluring idea: an easy way to see if editors are selecting the right content. But article citations and downloads won’t tell you if the peer review was robust. And there are lots of things that good editors do that wouldn’t be easily captured – for example, taking the time to offer constructive feedback when rejecting a paper, resolving publishing ethics cases, mentoring new Board members, etc. And like the impact factor, metrics would probably be biased in favor of fast-moving subject areas, and they could be open to manipulation!

Daniel: We’ve done a lot of research on this exact question over the past few months. A few main things stand out.

First, 89% of editors we surveyed wanted recognition for their handling editor work in the same way they now get recognition for their peer review work, so that they can provide proof of their editorial contributions in promotion applications. As a result of these requests Publons now supports recognition for handling editors.

Second, editors the world over all have the same answer for the toughest part of being an editor: finding good reviewers that accept review invitations. Journals can provide more support, tools, and training to help editors with finding reviewers -- and continue to improve the recognition and rewards for reviewers to improve review invitation acceptance rates.

There is a lot of work to be done in expanding the reviewer pool, most obviously by identifying (and providing peer review training for) early career researchers and researchers from non-Western countries.

Sam: I agree that one of the big challenges now is to expand the reviewer pool, especially in Asia, where we are seeing a rapid increase in publications in many subject areas. The ORCID initiative is going to help with this, but not everyone is participating yet. Journals and universities need to encourage their authors to register for ORCID identifiers to ensure that they get full credit for their publications.

Image credit: Colin Anderson/Getty Images

About the Author

Verity Warne

Author Marketing, Wiley // I am responsible for helping to market and develop products and services that will help improve authors' peer review experiences.

More Content by Verity Warne
Previous Article
Answering the Questions Behind Peer Review With PEERE
Answering the Questions Behind Peer Review With PEERE

Find out more about the PEERE project in this blog post from Michael Willis, Senior Manager in Wiley's Peer...

Next Article
What Can Publishers Do to Encourage and Contribute to a Culture of Transparency?
What Can Publishers Do to Encourage and Contribute to a Culture of Transparency?

Stephanie Loh and Hannah Wakley, peer review specialists at Wiley, discuss what publishers can do to encour...