[music]
Nandita Quaderi: I think we can already see some moves in place that are addressing the problems we have now. Some of these, for example, lie with research assessment exercises there is in regions that are still relying too much on bibliometric indicators to balance things out again. There is a move now to use more qualitative narrative expert opinion to balance out just using numbers. Again, that will help reduce these perverse incentives.
Intro: Ideas to Innovation from Clarivate.
Neville Hobson: Trust in what you read, whether it’s a report online or your daily newspaper in print has assumed extreme importance in recent years. Today, with so much information available online at the click of a mouse and increasingly at the tap of a finger, verification of information, who created it and who shared it has become part of standard operating procedures in many enterprises to create a climate of trust surrounding information you use and act on. Nowhere is this more true than in the field of scientific research and reporting.
Welcome to Ideas to Innovation season two. I am Neville Hobson. The annual Journal Citation Reports, which uses data from the Web of Science, a Clarivate product, is a great example of how quality data is a foundation from which we can demonstrate the integrity of that data that leads to user trust in information and insights derived from that data. In essence, research integrity is key to enabling the research community, publishers and librarians to evaluate and compare the scholarly impact of the world’s high quality journals using a range of indicators, descriptive data and visualizations.
To talk about this and gain insights into this important area, I’d like to welcome our guest Nandita Quaderi, who is Editor-in-Chief and Vice President of Editorial, Web of Science at Clarivate. Welcome Nandita.
Nandita Quaderi: Hello, Neville. Thank you. I’m very happy to be here.
Neville: You’ve played an active role in supporting research integrity throughout your career as a researcher, publisher and arbiter of quality. Can you tell our listeners a little about your role today at Clarivate?
Nandita: I joined Clarivate four years ago and in my role here as Editor-in-Chief and Editorial Vice President, I had overall responsibility for editorial strategy and for the policies and practices governing selection for the Web of Science and inclusion in the Journal Citation Reports. I started my career in academia. I have a degree in chemistry from Oxford and a PhD in molecular genetics from Imperial [College]. After a period as a post-doc in Italy, I returned to London to establish my own lab as a principal investigator at the MRC Centre for Developmental Neurobiology at Kings. I then moved into scholarly publishing and held roles including editorial director at BMC and publishing director for open research at Springer Nature.
Neville: Thanks Nandita. That sets the scene for what we’re going to talk about in this conversation I think. It puts you in a really good light. It qualifies it quite well, your background. Let’s consider our starting point. What does research integrity actually mean and why is it so important?
Nandita: Research integrity means conducting and presenting research in a way which allows others to have trust and confidence in the methods used, the results and the conclusions made based on the findings. The upholding of research integrity doesn’t just lie with researchers themselves. It’s a shared responsibility across all stakeholders in the research community. This includes institutes, funders, journal editors and peer reviewers and publishers.
Of course, it also includes companies such as Clarivate that provide data metrics and analytics. No discovery or innovation happens in a vacuum. We build on the foundations of previous work which spans across decades if not centuries. This is exactly what Sir Isaac Newton was referring to by his famous phrase, “If I have seen further, it is by standing on the shoulders of giants.” He wrote this in a letter to Sir Robert Hooke way back in 1675.
It’s absolutely vital that research and integrity is upheld and the scholarly record can be trusted to provide a solid foundation for science and scholarly activity to progress. Without a trusted record of research, it’s impossible to reliably build on previous ideas, replicate results or effectively utilize the outcomes of research. I’d like to give you an example of the dangers of not being able to trust the scholarly record if I may.
Neville: Please do.
Nandita: In 1998, Andrew Wakefield and colleagues published a paper in The Lancet claiming they’d found a link between the early childhood MMR vaccine for measles, mumps and rubella and autism. Despite the small sample size of only 12 and the speculative nature of the claims, the paper was widely publicized. As a result, MMR vaccination rates began to drop and these preventable childhood diseases re-emerged. Attempts to – -replicate these findings through much larger, large-scale studies were unsuccessful and Wakefield’s paper was eventually retracted in 2010.
Neville: That’s most interesting. That I suppose to my mind certainly highlights some fragility maybe in the chain, what you described at the start, talking about Sir Isaac Newton for instance and that historical foundational process leading to where we are today and then that happened that you just outlined. It seems quite fragile. Only as strong as its weakest link maybe. How do you prevent gaming the system by bad actors or those with little awareness and understanding? Can it be compromised? How if so?
Nandita: I think that’s a really nice way to put it. We are only as strong as our weakest link. Luckily, in general, the research community respects and upholds the values of honesty, rigor and transparency that underpin research integrity. Sometimes these values are undermined by lack of awareness or insufficient training in research and publication ethics and so better training and awareness of best practice would definitely help.
However, unethical behaviour can also be driven by the desire of some bad actors to gain an advantage at any cost, especially when there are systematic pressures in place. One example is the creation of perverse incentives through the overreliance and the misuse of bibliometric indicators in research assessment exercises. These indicators are used to decide how research funds are allocated and also in hiring and promotion decisions.
In the past, research assessment used to rely on qualitative assessments by experts. The introduction of quantitative data such as the number of publications, citation counts and indicators based on these numbers provided a complimentary approach that added objectivity that could counter any biases in qualitative assessment. We’ve now got to a point where in many regions only quantitative assessment of publication and citation numbers is used with no balancing expert qualitative assessment.
Evaluation based on the formulate use of data or just relying on single measures is not good practice. It creates perverse incentives and can result in unintended consequences as the scores or ranks become a goal in themselves. We see this in all walks of life. It’s often expressed as Goodhart’s law. When a measure becomes a target, it’s no longer a good measure. Unfortunately, this is what we are seeing in the arena of research assessment.
There’s always been a degree of fabrication, fortification and plagiarism. However, the overreliance on bibliometric indicators has created the perverse incentive to increase the number of publications and the number of citations as goals in themselves. What we see are new forms of manipulation emerge as some bad actors seek an unfair advantage and fraudulent enterprises that have appeared that exploit this increased pressure to publish and be cited. As a result, research integrity is being compromised and the scholarly record is being polluted.
Neville: That’s interesting. It doesn’t surprise me what you’ve outlined there Nandita. This I suspect reflects many other aspects of content, if I can just simply call it that, that’s published and shared particularly online. You mentioned plagiarism for example. That’s a huge issue generally speaking, is it not, hence the idea that you fact check things more than you ever did before and this is just the layperson. Someone like me for instance would be careful to check the validity, the veracity, if you will of content I’m reading particularly to trust it, who wrote it, where it’s published and who’s sharing it.
This doesn’t surprise me hearing what you’re saying. I’m thinking about the broad area of trust that is really enveloping our conversation here, that I mentioned at the start of the conversation. In light particularly what you mentioned on new forms of manipulation and all the rest and the damage that is a consequence of that, can you briefly share some ideas on how Clarivate addresses this problem in the scientific research area?
Nandita: Absolutely. As you say Neville, plagiarism and false data, fake news is everywhere in our lives but there’s some specific things that are limited to scholarly enterprises. Some examples of unethical manipulation that we are seeing in that arena include things like inappropriate self-citation, which means adding spurious citations to inflate citation counts and coercive citations when we see authors being pressured to add spurious citations by peer reviewers or journal editors.
We are also seeing these fraudulent enterprises spring up, and these include things like- -like paper mills that produce and sell fabricated manuscripts and provide fake peer reviews. We also have so-called predatory publishers that solicit manuscripts and charge a fee to publish without any kind of editorial oversight such as proper peer review. Given that the damage to research integrity is clear, you might well be wondering why do we still see the misuse of bibliometric indicators in research assessment?
Robert Shiller who won the 2013 Nobel Prize for Economics puts it like this. “People are often more willing to act based on little or no data than to use data that is a challenge to assemble.” We have an innate appeal in things like summary statistics and lead tables for research assessment, as they’re very quick and easy to use but what we see is metrics designed to measure a certain characteristic, being used to measure another characteristic.
For example, the Journal Impact Factor, which is a journal-level metric is being used inappropriately and irresponsibly as a measure for individual articles and researchers. Also, information is getting lost when data about researchers and their institutes are being squeezed into a simplified metric or a simple lead table.
Neville: Okay. That’s quite a background. It’s quite a minefield you are dealing with. I get a good understanding from what you said of how big that minefield is. Is this something that will always be with us, do you think?
Nandita: I think as long as we have these perverse incentives in place, there are always going to be people who are tempted to take a shortcut to get ahead.
Neville: Like much else in life, generally I suppose.
Nandita: Like everything else in life, exactly.
Neville: Yes. Can’t trust anything. You have to do your due diligence and check everything I think is the thing. If you are able to make that a simpler– or maybe not simpler, an easier process for the end user I suppose, is the key thing here, right?
Nandita: Absolutely. That’s where Clarivate really does play an important role. Research integrity can be compromised at many points in the research process and no one stakeholder can fix it. It’s a shared responsibility. At the Web of Science, we take our responsibility in upholding research integrity very seriously. We do this by providing our users with data and metrics they can trust. Every journal, book and conference proceeding that’s indexed in the Web of Science core collection has passed through a rigorous selection process by our team of expert in-house editors.
This means that our customers can rely on the Web of Science to provide a trusted source of content and of data in a world where we are seeing at ever-increasing levels of unethical and fraudulent behaviour. However, even the most trustworthy journals sometimes publish results that in the course of time turn out to be unreliable. It’s important that these results are flagged whether they be the result of unethical behaviour or poor practice, or just honest mistakes.
Publishers use corrections and retractions to flag erroneous results and to help maintain the integrity of the scholarly corpus, but there’s no universal standard practice for journals on how to handle or display retractions and so they could be missed by readers. Web of Science publication records and Clarivate’s EndNote reference management software now makes it clear when a publication has been retracted by adding a retraction ID to the publication or the reference. In EndNote, the Cite While You Write feature also shows whether a reference has been retracted or not. These are all ways in which Clarivate can help ensure that a flawed and misleading research is not cited and it’s not built upon.
Neville: That sounds pretty robust what you just explained. I want to ask you a bit about the expansion of the Jjournal Iimpact Ffactor, which I think fits nicely in the overall picture you’ve painted of integrity of research and doing more to I guess enhance the confidence of users and the build-up of trust. What can you tell us about the expansion? I understand it’s designed to help level the trust playing field for all quality scientific journals. Am I right with that?
Nandita: You’re absolutely right there, Neville. The Jjournal iImpact Ffactor, the JIF, is a measure of the overall scholarly impact of a journal. Currently only the most impactful science and social science journals indexed in this Web of Science core collection are given a JIF. This is published in a sister product called the JCR, the Journal Citation Reports. These journals have passed our rigorous quality evaluation criteria, and they’ve also passed our impact criteria. Our quality criteria are designed to select for editorial rigor and publication ethics, and only around 20%- -of journals that are submitted for evaluation pass these quality criteria and are included in the Web of Science.
Additionally, our impact criteria are designed to identify the most impactful, the most influential journals in a given discipline using citations as the primary determinant of impact. From next year’s JCR release, all journals indexed in the Web of Science core collection will be eligible for a JIF. What that means is that we’re expanding the JIF all the journals that have passed our rigorous quality criteria. Giving all quality journals a JIF will provide full transparency to articles and citations that have contributed to impact. Therefore, it’ll help them demonstrate their value to the global research community.
This will really help journals that apply high levels of editorial and publication ethics but aren’t always highly cited. This includes recently launched journals, opened access journals, journals with a niche or a regionally focused scope and journals from the global south. In next year’s JCR almost 9,000 journals from more than 3,000 publishers, many of which are smaller publishers from the developing world will have a JIF for the first time.
Neville: That sounds fabulous. I think you’ve set that out pretty well. It actually leads me nicely really to just a final question I’d like to ask you. Here we are in 2022, having this conversation and you’ve explained or indeed described a picture or painted a picture if you will, of a landscape that well, it’s certainly in flux in terms of trust. Bad actors, plagiarism, all the bad things amongst a lot of the good things such as you’ve outlined and what Clarivate is doing to enhance that feeling of trust by users in the data that they have access to and the reporting that derives from that.
What would you say Nandita, what’s your thinking on what are we looking at say just in five years’ time? That’s 2027, long before the end of the decade, which is 2030, what will this landscape look like then do you think?
Nandita: I think we can already see some moves in place that are addressing the problems we have now. Some of these, for example, lie with research assessment exercises there is in regions that are still relying too much on bibliometric indicators to balance things out again. There is a move now to use more qualitative narrative expert opinion to balance out just using numbers. Again, that will help reduce these perverse incentives.
There’s also a movement called the open research movement, which will help. The open research movement includes things like the open access publishing model, which removes subscription barriers, and so helps the visibility of the published research, but also things like open data and open peer review that provides even more transparency into the research lifecycle. I think having results under scrutiny at a much earlier stage and reducing reliance on that final published article will really help things along as well.
Neville: That’s terrific. I think you’ve painted– so I would translate everything you said essentially into it’s a work in progress, but there are very encouraging signs ahead, particularly in the areas Clarivate is working on. Is that a fair assessment do you think?
Nandita: I think that’s a good assessment. Absolutely.
Neville: Excellent. Okay. Well, Nandita, I’d like to thank you for your time and sharing your knowledge and insights about research integrity and building climates of trust. Thank you very much, indeed.
Nandita: Thank you, Neville. It’s been an absolute pleasure.
Neville: For more information visit clarivate.com and search research integrity. Season two of Ideas to Innovation continues with our next episode in a few weeks’ time. Visit clarivate.com/podcasts for information. Thanks for listening.
Voiceover: Ideas to Innovation from Clarivate.
[00:19:14] [END OF AUDIO]
