Need help finding peer reviewers who are reliable and motivated? Read Publons’ comprehensive guide for editors. We provide tips, advice, and the latest technology to find journal reviewers and avoid peer review fraud in today’s publish or perish culture.
Solutions for finding journal reviewers (at a glance):
- Recognize your peer reviewers
- Enhance review transparency at your journal
- Provide feedback and ask your authors to do the same
- Help train early-career researchers
- Improve your level of vetting
- Utilize new fraud detection tools in the market
- Benefit from Publons’ new reviewer selection software.
Reliable journal reviewers wanted… by everyone.
Looking for a motivated, reliable, and capable peer reviewer? Unfortunately, so is every editor… everywhere.
Finding, screening, and contacting peer reviewers has become a routine nightmare for editors worldwide. It’s the dark side of peer review that’s consumed many in a seemingly-endless search:
— Elisabeth Bik (@MicrobiomDigest) January 30, 2018
The rapid growth in manuscript submissions means editors are sending out more review invitations than ever before – and the review invitation acceptance rate for certain journals has dropped dramatically in recent years in response.
Editors now often have to find ten or more qualified potential reviewers to secure a couple of reviews. That’s time and effort few editors can afford to spare.
Raphael Didham, Editor of Insect Conservation and Diversity, found the median frequency of reviewers accepting the review invitations at his journal dropped noticeably from ~70% in 2008-2013 to ~50% in 2014-2016.
“We have had some manuscripts that have had as many as 15 reviewer invitations turned down, and I know of other journals with even more extreme examples of declining reviewer willingness to review.”
The rise of Peer Review Fraud
The peer review fraud playbook is simple: provide a real researcher’s name with a fake email under your control, wait for the editor to send a review invitation to the fake email address, and promptly write a glowing review of the obviously-great research you’ve contributed.
“When [editors] see eminent reviewers being suggested by fairly inexperienced researchers it is not unreasonable to invite them.”
Solutions to grow your list of reviewers
1. Reviewer recognition
Giving recognition for peer review is what we focus on here at Publons. And for good reason: it’s unrealistic for editors to continually ask time-pressured researchers to put aside their own work for peer review when that activity goes unrewarded.
Right now, most academics see performing peer review as a service to the community, but as we mentioned earlier, it’s a voluntary service that they fit in when they have the time. This can affect the quality (and perhaps the length) of their peer review.
This needs to change, and journal editors are uniquely placed to help drive this forward with the support of their publishers.
In a case study with the American Society for Microbiology (ASM), we were able to show that researchers are more willing to review and provide useful, constructive feedback if they know their contributions will be formally recognized.
Some options for peer review recognition at your journal include:
- publishing the names of your peer reviewers in your journal as a way to say thank you. Here’s an example from Wiley.
- reviewer rewards in the form of journal access or discounts.
- rewarding your reviewers with an authenticated certificate
- partnering with Publons (or encouraging your reviewers to join Publons and we’ll automate the process for you.
Read our series of case studies that demonstrate how peer reviewers use evidence of their peer review activity in their job and funding applications. Peer review helps researchers advance their careers and you can support them by validating their work.
Enhancing peer review transparency will shore up editors’ screening process.
Editors have limited insight into reviewer workloads. Peer review has historically been performed in silos and behind closed doors. It has led to little insight into who is shouldering the workload, what else they have on their plates right now, and what issues are discussed during the review process. It also lacks information about the quality of a reviewers’ contributions.
In our 2017 Reviewer Distribution Index, we found that in each country we measured, a reasonably small proportion of reviewers (10%-20%) were responsible for half of the reviews done in their country. You can read more about that here.
For editors, this can lead to several undesirable results, including unwittingly overloading the same (or same type) or researcher with multiple review requests or sending review invitations that fall outside of the reviewer’s expertise.
There have been numerous journal editors who have worked to learn more about their reviewers to shine a light on their own practices and the areas that need to be improved.
The American Geophysical Union did just that. Editors at the journal were interested to learn more about gender bias in scholarly peer review and so conducted a study that combined their member data with their editorial data. You can read more about their results here.
Peer reviewers spend countless hours enhancing the quality of scholarly communication, and editors and authors are uniquely placed to provide feedback on their helpfulness.
Publons’ feedback features for editors and open access authors helps to automate this feedback and recognizes the reviewers whose score demonstrates they’ve gone above and beyond in improving the research.
You can also work feedback into your own process at your journal by recognizing those who exceed expectations, and helping early-career researchers and others new to review by offering tips and advice for improvement.
4. Help train early-career researchers
Editors have told us sourcing reviewers is one of the hardest parts of their job.
At the same time, there is an abundance of early-career researchers keen to peer review, but simply haven’t been asked or don’t know how to connect with editors and prove their expertise.
There are numerous peer review training courses designed to bring more reviewers up to speed and grow editors’ reviewer pools. These include Nature’s Masterclass, Springer’s Author Academy, and our very own Publons Academy.
Publons’ peer review Academy is designed to tackle shrinking reviewer pools head on. It provides reviewers with a platform to demonstrate their reviewing expertise and connects them with the editors who need to grow their reviewer list.
The changes typically announced after the discovery of peer review fraud are that journals will adopt a higher level of vetting of potential reviewers, will stop accepting author-suggested reviewers, and/or will discourage against reviewers using non-institutional email addresses.
Taking these steps will help increase the chances of finding motivated and reliable reviewers. Unfortunately, however, each of the vetting solutions above also have limitations–namely in the form of time and resources.
Researchers can be nomadic — moving between institutions and using multiple email addresses. It makes an editor’s role in sourcing expert peer reviewers extremely difficult, especially at a time when manuscript submissions are rapidly increasing.
Editors can save themselves time and effort by connecting with reviewers via Web of Science Reviewer Locator. Web of Science Reviewer Locator offers a researcher’s latest contact email address from the Web of Science, and is generally connected to the researcher’s institution.
6. Utilize new fraud detection tools in the market
Mitigating the risk of reviewer fraud is a reputational risk for journal editors and publishers alike.
Editors can reduce this risk once the peer review has been submitted by encouraging their journals to adopt one of the new fraud detection tools on the market. These tools, which analyse multiple factors of a review to spot unusual activity, can alert the journal editor to a potential fake review and help determine the next steps.
Journals can also look to employ their own data integrity detectives or hire outside professional services. The idea of each approach is to spot an unusual peer review submission or activity warranting further investigation.
Listen to our podcast with Australian cancer researcher Dr. Jennifer Byrne (University of Sydney) who analyzes childhood and adult cancers at a molecular level by day, and by night, weeds out problematic cancer research with her fraud detection tool, Seek & Blastn.
All of the steps above will go a long way to speeding up the peer review process for researchers and editors. They will not, however, stop peer review fraud entirely — which means the editor and publisher still face a very real risk in the search, screen, and contact phases of peer review.
Editors need access to powerful and innovative recruitment tools that help them find and contact motivated, reliable, and capable peer reviewers, and avoid reputational risk for the publisher.
Web of Science’s Reviewer Locator does just that. Publishers subscribing to the service gain access to more than seven million expert researchers, which draws from a combination of Publons’ unique database of reviewers and the world’s premier article and citation index, Web of Science.
Web of Science Reviewer Locator provides a 360-degree view of researchers to offer editors the data they need to make more informed reviewer decisions. Editors can search and filter for reviewers using article titles and full abstracts, keywords, co-authors, and as the software develops, reviewer location. They will then be privy to a range of possible experts, with the opportunity to view their review and publication history, editorial contributions and board memberships, and their current availability to review.
What do you think? If you have more tips to add or advice for editors share it with us in the comments below or at email@example.com.