Evaluation and selection process for Highly Cited Researchers

DAVID PENDLEBURY

Head of Research Analysis at the Institute for Scientific Information

Clarivate

Every year, the unveiling of the Highly Cited Researchers™ list sparks anticipation within the global research community as both researchers and their affiliated institutions or organizations scrutinize the list. Today, we present our 2023 list, recognizing 6,849 individuals from more than 1,300 institutions across 67 countries and regions.

These Highly Cited Researchers stand out for their significant and broad influence in various scientific and social science domains, representing a small fraction of the researcher population whose contributions disproportionately extend the frontiers of knowledge and gain societal innovations that make the world healthier, more sustainable and more secure.

Our 2023 evaluation and selection policy

As the need for high-quality data from rigorously selected sources is becoming ever more important, the Institute for Scientific Information (ISI)™ has had to adapt and respond to technological advances and changes in the publishing landscape when selecting individuals for inclusion. Just as we have applied stringent standards and selection criteria to identify trusted journals in the Web of Science™, we have evolved our evaluation and selection policies for our annual Highly Cited Researchers program to address the challenges of an increasingly complex and polluted scholarly record.

Our process for identifying individuals to our Highly Cited Researchers list includes the consideration of many qualitative and quantitative factors. The published list is determined by combining the inter-related information available to us.

We start with the creation of a new preliminary list each year based on our citation data from the papers published from the previous 11-year rolling window. We then refine it with the information that lies beneath, using qualitative analysis and expert judgement.

After the citation triage stage, we look at a growing number of factors when evaluating papers including, but not but limited to:

• Hyper-authorship of papers – Our expectation is that an author has provided a meaningful contribution to any paper which bears their name and the publication of multiple papers per week over long periods strains our understanding of normative authorship and credit.
• We exclude papers which reveal an excessive level of self-citation for the field of research – our paper outlines how we determine How much is too much? 
• We look for evidence of extraordinary, very recent publications that represent research of incremental value, accompanied by high levels of author self-citation.
• Unusual patterns of collaborative group citation activity and anomalous levels of citations from co-authors. The identification of networks of co-authors raises the possibility that an individual’s high citation counts may be highly reliant on citations from this network; if more than half of a researcher’s citations derive from co-authors, we consider this to be narrow influence, rather than the broad community influence we seek to reflect.

With the continuing assistance of Retraction Watch and its unparalleled database of retractions we enhanced our qualitative analysis in 2022; this exercise has continued. This year we also took account of expressions of concern from identified representatives at research institutes, national research managers and our institutional customers, along with information shared with us by other collective community groups, e.g. For Better Science, Pub Peer. Some of these resources include anonymous or whistleblower sources. We also consider such evidence from trusted sources – where we can verify claims through direct observation.

By implementing an increasing number of qualitative criteria and markers this year, the number of preliminary candidates excluded from our final list increased from 500 in 2022 to more than 1,000 for 2023.

Click on the image to expand.

Learn more about our Highly Cited Researchers evaluation and selection process on our website.

Exclusion of ESI field of Mathematics

We have excluded the Mathematics category from our analysis for 2023. The field of Mathematics differs from other categories in Essential Science Indicators (ESI)™. It is a highly fractionated research domain, with few individuals working on a number of specialty topics. The average rate of publication and citation in Mathematics is relatively low, so small increases in publication and citation tend to distort the representation and analysis of the overall field. Because of this, the field of Mathematics is more vulnerable to strategies to optimize status and rewards through publication and citation manipulation.

The responsible approach now is to afford this category additional analysis and judgement to identify individuals with significant and broad influence in the field. We are consulting with leading bibliometricians and mathematicians to discuss our future approach to the analysis of this field.

Recognizing Highly Cited Researchers 2023

We seek to use stringent standards and clear evaluation and selection criteria for our Highly Cited Researchers program as we celebrate exceptional and broad influence of individuals in their field or fields of research.