It’s not always easy to spot flaws in research papers. Sometimes an error is glaringly obvious – like a vague abstract with no aim and little data – and other times it’s like finding a needle in a stack of pins. (Like, really, really sharp pins that leave you dreaming of haystacks). Luckily, the solution isn’t all that prickly. The trick is knowing what to look for during peer review, where to find it and, importantly, how severe the error is.
To help with this, we’ve pulled together a list of six common flaws you can watch out for as a reviewer.
Here are the 6 common flaws to look out for in peer review:
1) Inappropriate study design for the study aims
2) Unexplained deviations from standard/best practice and methodologies
3) Over-interpretation of results
4) Commenting beyond the scope of the article
5) Lack of evidence to support conclusions
6) Too many words
This blog post is informed by a module within our free peer review training course, the Publons Academy. Join today to learn the core competencies of peer review. With one-to-one support from your mentor, you’ll write reviews of real papers, gain access to our Review Template and review examples, and by the end of it, we’ll put you in front of journal editors in your field as a certified peer reviewer.
We discuss each common flaw to watch out for below, but first, a quick recap about the importance of peer review and how it can improve your own research.
Why peer review is important
As a peer reviewer, you play a vital role in protecting the quality and integrity of scientific research. Your peers rely on this work to understand what research to trust and build on, leading to better, faster science.
Your manuscripts will also improve because, over time, you’ll learn how to use your knowledge in peer review to fine-tune your own research.
Peer reviewing will help you evaluate the importance and accuracy of your research question; the appropriateness of methodological and statistical approaches; and build up a set of best-practice tips to prepare and organize your research project. And finally, by learning common errors to watch out for when peer reviewing, you’ll inevitably learn to avoid the same mistakes in your own research, which will increase your chances of getting published.
6 Common Research Flaws and How to Spot them in a Manuscript
A quick caveat: this isn’t an extensive list! Research errors differ for every field as do the types of studies conducted. This is a helpful starting point, however, that will enable you to guard against the more common mistakes made in a manuscript. Once you start accepting more invites to review and become more confident reading a manuscript critically, you can build on this list with more specialized examples.
1. Inappropriate study design for the study aims
A study’s design is crucial to obtaining valid and scientifically sound results. Familiarise yourself with those commonly used in your field of research. If you come across an uncommon study design, read the researchers’ use and justification of it carefully, and question how it might affect their data and analysis. Review the study design critically but also remember to be open-minded. Just because something is new and unfamiliar it does not automatically mean it is incorrect or flawed.
2. Unexplained deviations from standard/best practice and methodologies
Similar to the above. The methods section, for example, should explain the steps taken to produce the results. If these are not clear or you’re left questioning their validity, it’s important to make your concerns known. And if they are unusual then, as with the study design, examine the researchers’ justification carefully with the view to ask more questions if necessary. Non-academic discourse, whereby opinionated and biased statements are used throughout the study, is another deviation from best practice
3. Over-interpretation of results
Over-interpretation has no place in research. Ensure the conclusions drawn in the paper are based on the data presented and are not extrapolated beyond that (to a larger population or ecological setting, for example). You should also watch out for studies that focus on seemingly important differences where none exist.
4. Commenting beyond the scope of the article
“That’s beyond the scope of this paper” is a common phrase in academic writing. As a reviewer, watch out for papers that include comments or statements not pertaining to the research project and data at hand.
5. Lack of evidence to support conclusions
A research paper’s concluding statements must be justified and evidence-based. If you’re not convinced of the results, it could mean the researchers need to clarify aspects of their methodological procedure, add more references to support their claims, or include additional data or further analysis.
6. Too many words
A common pain point in manuscripts is that it’s too wordy. It’s important to keep check on this scenario and encourage clear, concise and effective text where possible. Too many words can be distracting for the reader, which at best could cause them to lose interest and at worst could lead to them misinterpreting the research.
If you’re interested in learning more about common flaws and how to address them in peer review, sign up for our Publons Academy. You’ll gain practical experience with this free, on-demand course by writing real reviews with one-to-one support for your mentor. After completion, you’ll also be put in front of editors at elite journals in your field.
Want to learn more? See our 12 step guide about reviewing a manuscript critically.
Publons allows you to record, verify, and showcase your peer review contributions in a format you can include in job and funding applications (without breaking reviewer anonymity).
Register now to start building your verified peer review record.
