The Wiley Network

How an Editor-in-Chief’s Location Influences the Regional Distribution of Reviewers

how-an-editor-in-chief-s-location-influences-the-regional-distribution-of-reviewers

Thomas Gaston, Peer Review Management, Wiley

April 01, 2018

The difficulty of finding willing reviewers is well known. This is indicated in the invitation acceptance rate and the time taken by editors to secure reviewers. The reasons for these difficulties are many and varied, but may relate to a variation between those researchers writing papers and those being invited to review. One well known imbalance is that a large number of review invitations are directed to a small number of people; senior researchers can find themselves bombarded with requests to review, whilst many early career researchers are yet to be invited. Another imbalance is regional. In 2015 Wiley undertook research (Warne, 2016) which, along with other similar studies (e.g. Kovanis, 2016, and Mulligan and van Rossum, 2014), found an uneven burden upon researchers from the USA, who provide 33-34% of reviews compared to 22-24% of submissions.

As a member of the Peer Review Management team at Wiley where we endeavor to lead best practice, I wanted to understand the reasons for this regional imbalance. Previous research found that non-US researchers are willing to review (Mulligan and van Rossum, 2014); the problem seems to be that they are not being invited. Pippa Smart (Editor-in-Chief of Learned Publishing) and I decided to investigate the factors involved in the selection of reviewers and their subsequent agreement to review. Our hypothesis was that there would be a correlation between the location of the Editor-in-Chief (EiC) and the location of the reviewer. We also looked at other potential factors, including the location of the author, the ranking of the journal, the size of the journal, and the apparent difficulty the journal had in obtaining reviews.

We selected two subject areas to look at, Medicine and Agricultural & Biological Sciences, and then downloaded the data from ScholarOne Manuscripts. The data was anonymized and then analyzed. Our research evaluated 149 journals, involving 55,732 articles and 208,084 invitations to review to 110,053 reviewers. Of these invitations, 105,235 (51%) were accepted, 57,181 (27%) were declined and 45,668 (22%) received no response. 

Our findings endorsed earlier investigations that have revealed an imbalance between the locations of the corresponding authors (as declared in the author affiliation) and the locations of reviewers. In total (the entire data set), 25.68% of authors come from Asia, but only 9.08% of reviewers. 33.23% of authors come from Europe, but only 27.97% of reviewers. 23.85% of authors come from North America, and 31.7% of reviewers.

In addition, we found that EiCs select reviewers from their own region more often than from other regions (with the exception of EiCs in Oceania). The finding was repeated at a country level; EiCs more frequently select reviewers from their own country than the average. This is particularly notable for the Scandinavian countries, USA, Germany, and UK. EiCs from the USA, for example, select 50% of reviewers from the USA, compared to the average of 38%; and EiCs in the UK select UK-based reviewers twice as often as the average (22% compared to 11%). However, all EiCs show a preference for US-based reviewers.

The other factor where we found a clear correlation was between the location of the corresponding author and that of the reviewer. This was particularly notable for articles with Chinese, American, and Iranian corresponding authors. At a regional level, it seems that a very high proportion of those papers being sent to reviewers in Asia have Asian corresponding authors (55.9%). Similarly, reviewers in Africa are being sent a high proportion of papers with African corresponding authors. So, whilst there seems to be an overall preference for reviewers from the USA (a greater proportion of review invitations are sent to reviewers based in the USA), there is also a preference for sending papers to reviewers from the same region and country as the authors.

We also found some indications that reviewers were more likely to accept invitations, and more likely to give positive reviews, in cases where they were from the same region as the corresponding author. The effect was small and would warrant further research. None of the other factors we looked at seemed to correlate significantly with the location of the invited reviewer.

Given these findings, the question is why editors prefer reviewers from their own location and/or from the same location as the author. Our intuition is that this is due to the way editors select reviewers, which (anecdotally) is still primarily from their own networks rather than favoring other reviewer-finding strategies, such as following citations or automated searches. There may be other factors as well. One way of assessing the suitability of an unfamiliar reviewer is by looking at the institution they are affiliated with; if editors are unfamiliar with institutions in other regions they may feel uncertain about using that reviewer. Our results also may suggest that editors may purposefully select reviewers from the same location as the author, which would make sense in cases where there is a geographically specific component to the research.

There is no obligation on editors to have an even geographic distribution of reviewers; editors should always select reviewers based upon their appropriateness. However, given that there is a pre-existing imbalance in the burden of peer review, it is advisable to broaden the pool of reviewers to include as many relevant individuals as possible. Our results suggest this is not currently happening in all cases. We would encourage editors to search for reviewers beyond their own networks to more evenly distribute the burden of review.

Our research is published in Learned Publishing and can be found on Wiley Online Library.

References
Gaston, T., & Smart, P. (2018). What influences the regional diversity of reviewers: A study of medical and agricultural/biological sciences journals. Learned Publishing, http://bit.ly/2GggI8b
Kovanis, M., Porcher, R., Ravaud, P., & Trinquart, L. (2016). The global burden of journal peer review in the biomedical literature: Strong imbalance in the collective enterprise. PLoS ONE, 11(11), e0166387.  https://doi.org/10.1371/journal.pone.0166387
Mulligan, A., & van Rossum, J. (2014). What researchers think about the peer-review process. Retrieved from https://www.elsevier.com/editors-update/story/peer-review/what-researchers-think-about-the-peer-review-process
Warne, V. (2016). Rewarding reviewers - Sense or sensibility? A Wiley study explained. Learned Publishing, 29(1), 41–50. https://doi.org/10.1002/leap.100

Related Articles