The emerging field of Forensic Scientometrics (FoSci)

An introduction

Several major news stories have covered journal editors being bribed, university presidents resigning for questionable research integrity standards, and undisclosed conflicts of interest by faculty and researchers. Each day, it seems a new story about lapses in research integrity in science is published, which sometimes brings to light those individuals and groups discovering the misconduct. This work of inspecting and upholding the integrity of scientific research has long been conducted in the background of science and scholarly communication, carried out by passionate individuals driven by ethics and principles to ensure the veracity of the scientific method or record and the downstream impacts on policy, practice, and theory. 

Groups and individuals, such as Retraction Watch, have monitored, collected, classified, and written about retractions for over 10 years. Many of those doing the verifying have specialization detection — from nefarious networks to image manipulation to tortured phrases. Additionally, many organizations have developed specific offices and infrastructure to support research integrity — such as the US Office of Research Integrity at the National Institutes of Health, institutionally based research integrity and security offices, as well as the newly formed research integrity offices located in major publishing organizations.  

Despite the growth of investigative research and methods on scientific misconduct, the discipline itself lacks a common definition and description of its field. So, what do you call the collective work of analyzing publications, data, images, and statistics to uncover errors, misinformation, or manipulations in scientific publications? We propose calling this emerging field — forensic scientometrics — FoSci for short.

Why use forensics?  Forensics refers to applying scientific knowledge and methods to matters of law, particularly for investigating crimes and providing evidence in legal proceedings.

First, there is an investigative nature of the work we do even if the results do not end up in the court of law. While fraud investigations and syndicate networks encompass legal realms, the intentional manipulation of anything scientific (as in the process of scientific discovery or manipulation) does not have a special place. That doesn’t mean it shouldn’t be there.

Second, scientific publications are being used in the court of law as evidence, but the papers themselves and their veracity do not get scrutinized by expert scientometricians. A common belief is that a peer-reviewed academic paper indicates that the research has passed the scientific method stress test. Yet, peer-reviewers vet (or should) the scholarly question within the paper, not the weight of evidence in a societal context. 

Scientometrics involves the quantitative analysis of scientific publications and research outputs in this larger context. It encompasses the measurement and evaluation related to scientific activities, such as the impact of research, patterns of collaboration among researchers, citation analysis, and the productivity assessment and influence of individuals, institutions, or scientific journals. Scientometrics employs statistical and mathematical methods to derive meaningful insights into the structure and dynamics of scientific knowledge, contributing to our understanding of the scientific community's development and impact over time.

As we navigate the evolving landscape of scientific inquiry, the emergence of forensic scientometrics as a distinct field reflects a collective commitment to upholding research integrity; from the pioneers who have tirelessly exposed misconduct to the institutional changes taking place, the journey towards a recognized field is well underway. In this era of increased scrutiny, defining and embracing forensic scientometrics — FoSci — becomes essential in strengthening trust in and around science.

[Originally posted in May 2024 here.]

BIO
Leslie D. McIntosh, PhD is the VP of Research Integrity at Digital Science and dedicates her work to improving research, reducing disinformation, and increasing trust in science.

As an academic turned entrepreneur, she founded Ripeta in 2017 to improve research quality and integrity. Now part of Digital Science, the Ripeta algorithms detect Trust Markers of research manuscripts. She works with governments, publishers, institutions, and companies around the globe to improve research and scientific decision-making. Dr. McIntosh’s work was the most-read RetractionWatch post of 2022. In 2023, her influential ideas on achieving equity in research were highlighted in the Guardian and Science.