Over the last decade, research evaluation has shifted focus, not from quality ― which remains essential ― but from academic impact to societal and economic impact. Here we outline what’s ahead for the Web of Science™: transforming to a highly dynamic, personal and indispensable research assistant that support researchers’ needs across their workflows.
Sixty years ago, when the Institute for Scientific Information (ISI) was originally established by Dr. Eugene Garfield, scholarly literature was something that was handled, leafed through, discovered in libraries, delivered in the post, scribbled on, left on a desk and probably read in a comfortable chair. Since then, the digital content transformation has brought the literature online, where it is keyword searched, scrolled, bookmarked, shared, reviewed and can be read in bed or on the bus.
Through this period, the Web of Science has evolved as the world’s leading citation index. The Science Citation Index (launched 1964) and later Social Sciences Citation Index (1973) were originally designed as tools for literature discovery and indexing, and were commonly used by researchers and librarians to connect people with relevant scholarly content – a fundamental use case that is still realized today on the Web of Science by 150,000 researchers daily, albeit digitally.
In the 1990s, the Web of Science gained value as the basis of an analytical database as the use of bibliometrics in research evaluation became more popular. As time goes on, more and more stakeholder groups find value in the Web of Science data as a means to assess their role in the research ecosystem, be they publishers, research managers, researchers, librarians, funders, policy makers, private companies, non-profit organizations and more.
So what does this mean for the broader research community, and for the Web of Science?
An increasingly collaborative and connected future
We envision a world where the entire research ecosystem is more collaborative and connected, where researchers have faster, more intuitive access to the information, insights and tools needed to assess research needs, secure funding, and conduct, socialize and bring research to life so that it benefits society.
The future Web of Science will move beyond a passive search and discovery tool to a highly dynamic, personal and indispensable research assistant that support researchers’ needs across their workflows.
The transformation to digital literature, as well as many other online research activities, paves the way for our vision to become a reality and brings enormous benefit to the researcher.
This is increasingly important because, over the last decade, research evaluation has shifted focus, not from quality ― which remains essential ― but from academic impact to societal and economic impact.
In the past, evaluation was focused on research outputs (mainly articles and books) and on ways to measure their implicit quality in respect to local and international benchmarks using citation analysis. However, accountability and scrutiny have become more important for publicly funded research. It is crucial to show the wider return on investment in basic research, and the fact that socio-economic benefits and other outcomes (impact) are now documented during the evaluation process enables this transparency.
Since bibliometric indicators became a staple in research evaluation for measurements of quality, we now look to datasets that might become the source of complementary metrics to inform impact assessment.
There are obvious places to start: citations from policy documents and other grey literature provide evidence of relevance to policy setting; patent databases provide information on invention; clinical trials show the progression of medical research; and registration of spin-out companies and private-sector collaboration reflect industry impact and knowledge exchange. Each new source provides unique challenges, both in terms of acquisition and interpretation of the data but, in time, incorporation of these data sources will provide a broader basis upon which to build indicators of impact.
The breadth and depth of data that can inform us about research will continue to broaden. Its scope will grow to cover a wider range of research activities through an increased diversity of content types, supplying the basis for a more expressive platform for researchers to profile their work. It will be a community driven effort – no single authority will arbitrate all of these data as it will be produced across many different silos. However, crucial infrastructure will be required to enable seamless integration, and that is where the Web of Science provides unique competency and value.
Selectivity and neutrality have always been fundamental principles for the Web of Science. In our mission to create a record of lasting value for the most important research from around the world, the Web of Science has become the trusted and accurate data source on reputable research. For example, the criteria required for inclusion in the Web of Science Core Collection mean that only journals satisfying a series of scholarly quality standards are added. When producing the annual list of Highly Cited Researchers, manual curation is used to ensure that all relevant publications for individuals are properly attributed. This is why key ranking organizations make use of our data, and it has become embedded in national and institutional research information systems all over the globe.
Critical success factors
Herein lies the challenge and opportunity that are essential to our vision: to extend the breadth of data brought into the broader ecosystem while maintaining high editorial standards, ensuring robust and appropriate metrics are devised and then supported with interpretive guidance.
It is not a new idea to combine multiple data sets about the research ecosystem (e.g., grants, publications, patents, clinical trials), but there are several major elements to address. These include building good metrics to describe and characterize research impact, and providing a platform that allows researchers to join and index relevant content on their research activities from trusted sources.
Against the background of a broader and more sophisticated data system, further advances in the area of discovery and data visualization pave the way to modernizing research conduct and measurement. Today, most researchers find literature using search technology that is directly adopted from the web – they look for keywords in documents and sift through a list of results sorted according to a relevancy indicator. There are many ways in which this paradigm can be improved through use of the citation network and data visualization tools, and we are actively incorporating them as we enhance the Web of Science platform.
Science mapping – the ability to convey large-scale landscapes of research topics – benefits from recent advances in machine learning and computational capacity, and when based on the explicit signals present in the citation network, provides a solid foundation to position research in a wider visual context. This capability will become especially important as more and more information on the research process is brought together. Overlaying information on grants awarded, research capacity, articles published, industrial collaboration and the impact of research using common visualisation tools is a tantalizing idea that requires considerable expertise and a lot of data wrangling. Fortunately, these are some of the core capabilities that position us well to make our vision a reality in the very near term.
Given some of the upcoming releases to the Web of Science platform as well as other new capabilities planned for development, we are well on our way to blaze a trail for the next 60 years. We hope that Dr. Garfield would be proud, and are honored to maintain his legacy.
Watch this space for news about our upcoming webinar plus podcast series, ‘Reimagining the future of research evaluation and impact measurement.’