Blog navigate_next Academia and Government

Editors’ guide to finding peer reviewers

Share this article
8 minute read

Need help finding peer reviewers who are reliable and motivated? Read Publons’ comprehensive guide for editors. We provide tips, advice, and the latest technology to find journal reviewers and avoid peer review fraud in today’s publish or perish culture.

Solutions for finding journal reviewers (at a glance):

Reliable journal reviewers wanted… by everyone.

Looking for a motivated, reliable, and capable peer reviewer? Unfortunately, so is every editor… everywhere.

Finding, screening, and contacting peer reviewers has become a routine nightmare for editors worldwide. It’s the dark side of peer review that’s consumed many in a seemingly-endless search:

 

 

The rapid growth in manuscript submissions means editors are sending out more review invitations than ever before – and the review invitation acceptance rate for certain journals has dropped dramatically in recent years in response.

Editors now often have to find ten or more qualified potential reviewers to secure a couple of reviews. That’s time and effort few editors can afford to spare.

Raphael Didham, Editor of Insect Conservation and Diversity, found the median frequency of reviewers accepting the review invitations at his journal dropped noticeably from ~70% in 2008-2013 to ~50% in 2014-2016.

 

“We have had some manuscripts that have had as many as 15 reviewer invitations turned down, and I know of other journals with even more extreme examples of declining reviewer willingness to review.”

Raphael Didham, Editor of Insect Conservation and Diversity in an interview with Retraction Watch.
Web of Science Reviewer Locator can help tackle this problem head on. It helps journal editors find, screen, and connect with peer reviewers who are fit-for-purpose, available, and motivated – while mitigating the risk of reviewer fraud. It does this by combining the power of Publons’ exclusive peer review database with the unparalleled Web of Science author and citation index.
We talk about Web of Science Reviewer Locator a bit later on in this guide, but first: let’s take a look at where the problems first started for editors in peer review and what it’s led to.

Publish or perish and the problems with peer review

Peer review is, as Henry Rzepa, chemist and Emeritus Professor at Imperial College London put it, “the single most important activity a publisher can undertake on behalf of the scientific community”. It is the gold standard of maintaining trust and efficiency in published research–but it’s far from perfect.

Peer review is failing to cope with today’s highly-competitive academic market because it hasn’t scaled with time.

Few researchers have the capacity to accept and submit quality and timely peer reviews in academia’s “publish or perish” culture, whereby successful funding, promotions, and grants applications favor publication in prestigious, high impact factor journals.

The Academic Publishing Machine is long overdue for evolution. There are few incentives to encourage researchers to put aside their teaching, mentoring, and scientific activities for review – despite it being at the center of trust and integrity of published research.

Until such incentives are put in place, peer review will always struggle to compete with the other priorities of a researcher. And until such incentives are put in place, editors will always struggle with:

  • Researchers who don’t respond to a peer review invitation
  • Researchers who accept a peer review invitation but are never heard from again.
  • Researchers who accept the peer review invitation but are so busy with their other commitments they submit a low-quality review.
  • Researchers who accept the peer review invitation and submit a high-quality review.

The worst scenario today’s publish or perish culture brings to scholarly research, however, is:

  • A researcher who accepts the peer review invitation but is not who they say they are.

This is peer review fraud. It is one of the most publicized problems with peer review as it continues to make headlines worldwide.

 

The rise of Peer Review Fraud

The peer review fraud playbook is simple: provide a real researcher’s name with a fake email under your control, wait for the editor to send a review invitation to the fake email address, and promptly write a glowing review of the obviously-great research you’ve contributed.

 

“When [editors] see eminent reviewers being suggested by fairly inexperienced researchers it is not unreasonable to invite them.”

Adam Cohen, Editor of Wiley’s British Journal of Clinical Pharmacology.
In an editorial for Wiley’s British Journal of Clinical Pharmacology, Adam Cohen describes how he and his editorial team were victims of a peer review ring. The authors or the company the authors paid for put forward seemingly-reliable names and bogus email addresses. They then approved the paper with little or no changes.With more than 500 cases to peer review fraud exposed to date, it is severely undermining the efforts of genuine reviewers, fracturing the public’s trust in research, and slowing down breakthrough discoveries.Read more about peer review fraud and the need to end it in our blog post.A more robust and effective peer review process is essential to maintain the quality and integrity of published research. An essential part of this process involves ensuring editors are supported in managing peer review, and can quickly find, screen, and connect with the best reviewers, every time.We’ve put together some tips and advice on how best to achieve that, starting with what we consider one of the most important changes needed in peer review: reviewer recognition.

 

Solutions to grow your list of reviewers

1. Reviewer recognition

Giving recognition for peer review is what we focus on here at Publons. And for good reason: it’s unrealistic for editors to continually ask time-pressured researchers to put aside their own work for peer review when that activity goes unrewarded.

Right now, most academics see performing peer review as a service to the community, but as we mentioned earlier, it’s a voluntary service that they fit in when they have the time. This can affect the quality (and perhaps the length) of their peer review.

This needs to change, and journal editors are uniquely placed to help drive this forward with the support of their publishers.

In a case study with the American Society for Microbiology (ASM), we were able to show that researchers are more willing to review and provide useful, constructive feedback if they know their contributions will be formally recognized.

American Society for Microbiology

 

Some options for peer review recognition at your journal include:

  • publishing the names of your peer reviewers in your journal as a way to say thank you. Here’s an example from Wiley.
  • reviewer rewards in the form of journal access or discounts.
  • rewarding your reviewers with an authenticated certificate
  • partnering with Publons (or encouraging your reviewers to join Publons and we’ll automate the process for you.

Read our series of case studies that demonstrate how peer reviewers use evidence of their peer review activity in their job and funding applications. Peer review helps researchers advance their careers and you can support them by validating their work.

 

2. Enhance transparency in review

Enhancing peer review transparency will shore up editors’ screening process.

Editors have limited insight into reviewer workloads. Peer review has historically been performed in silos and behind closed doors. It has led to little insight into who is shouldering the workload, what else they have on their plates right now, and what issues are discussed during the review process. It also lacks information about the quality of a reviewers’ contributions.

 

 

In our 2017 Reviewer Distribution Index, we found that in each country we measured, a reasonably small proportion of reviewers (10%-20%) were responsible for half of the reviews done in their country. You can read more about that here.

For editors, this can lead to several undesirable results, including unwittingly overloading the same (or same type) or researcher with multiple review requests or sending review invitations that fall outside of the reviewer’s expertise.

There have been numerous journal editors who have worked to learn more about their reviewers to shine a light on their own practices and the areas that need to be improved.

The American Geophysical Union did just that. Editors at the journal were interested to learn more about gender bias in scholarly peer review and so conducted a study that combined their member data with their editorial data. You can read more about their results here.

 

3. Provide feedback and encourage your authors to do the same

Peer reviewers spend countless hours enhancing the quality of scholarly communication, and editors and authors are uniquely placed to provide feedback on their helpfulness.

Publons’ feedback features for editors and open access authors helps to automate this feedback and recognizes the reviewers whose score demonstrates they’ve gone above and beyond in improving the research.

You can also work feedback into your own process at your journal by recognizing those who exceed expectations, and helping early-career researchers and others new to review by offering tips and advice for improvement.

 

4. Help train early-career researchers

Editors have told us sourcing reviewers is one of the hardest parts of their job.

At the same time, there is an abundance of early-career researchers keen to peer review, but simply haven’t been asked or don’t know how to connect with editors and prove their expertise.

There are numerous peer review training courses designed to bring more reviewers up to speed and grow editors’ reviewer pools. These include Nature’s Masterclass, Springer’s Author Academy, and our very own Publons Academy.

Publons’ peer review Academy is designed to tackle shrinking reviewer pools head on. It provides reviewers with a platform to demonstrate their reviewing expertise and connects them with the editors who need to grow their reviewer list.

Find out more about the Publons Academy here, or register to be a mentor for new reviewers here.

 

5. A higher level of vetting

The changes typically announced after the discovery of peer review fraud are that journals will adopt a higher level of vetting of potential reviewers, will stop accepting author-suggested reviewers, and/or will discourage against reviewers using non-institutional email addresses.

Taking these steps will help increase the chances of finding motivated and reliable reviewers. Unfortunately, however, each of the vetting solutions above also have limitations–namely in the form of time and resources.

Researchers can be nomadic — moving between institutions and using multiple email addresses. It makes an editor’s role in sourcing expert peer reviewers extremely difficult, especially at a time when manuscript submissions are rapidly increasing.

Editors can save themselves time and effort by connecting with reviewers via Web of Science Reviewer Locator. Web of Science Reviewer Locator offers a researcher’s latest contact email address from the Web of Science, and is generally connected to the researcher’s institution.

 

6. Utilize new fraud detection tools in the market

 

Mitigating the risk of reviewer fraud is a reputational risk for journal editors and publishers alike.

Editors can reduce this risk once the peer review has been submitted by encouraging their journals to adopt one of the new fraud detection tools on the market. These tools, which analyse multiple factors of a review to spot unusual activity, can alert the journal editor to a potential fake review and help determine the next steps.

Journals can also look to employ their own data integrity detectives or hire outside professional services. The idea of each approach is to spot an unusual peer review submission or activity warranting further investigation.

Listen to our podcast with Australian cancer researcher Dr. Jennifer Byrne (University of Sydney) who analyzes childhood and adult cancers at a molecular level by day, and by night, weeds out problematic cancer research with her fraud detection tool, Seek & Blastn.

 

7. Benefit from Publons’ new reviewer selection software

All of the steps above will go a long way to speeding up the peer review process for researchers and editors. They will not, however, stop peer review fraud entirely — which means the editor and publisher still face a very real risk in the search, screen, and contact phases of peer review.

Editors need access to powerful and innovative recruitment tools that help them find and contact motivated, reliable, and capable peer reviewers, and avoid reputational risk for the publisher.

Web of Science’s Reviewer Locator does just that. Publishers subscribing to the service gain access to more than seven million expert researchers, which draws from a combination of Publons’ unique database of reviewers and the world’s premier article and citation index, Web of Science.

Web of Science Reviewer Locator provides a 360-degree view of researchers to offer editors the data they need to make more informed reviewer decisions. Editors can search and filter for reviewers using article titles and full abstracts, keywords, co-authors, and as the software develops, reviewer location. They will then be privy to a range of possible experts, with the opportunity to view their review and publication history, editorial contributions and board memberships, and their current availability to review.

What do you think? If you have more tips to add or advice for editors share it with us in the comments below or at marketing@publons.com.