- A new study shows that artificial intelligence systems used to diagnose cancer from pathology slides do not perform equally for all patients, with accuracy varying across different demographic groups.
- Researchers pinpointed three key reasons behind this bias and created a new approach that significantly reduced these differences.
- The results emphasize why medical AI must be routinely evaluated for bias to help ensure fair and reliable cancer care for everyone.
Pathology and the Foundations of Cancer Diagnosis
For decades, pathology has been essential to how doctors diagnose and treat cancer. A pathologist studies an extremely thin slice of human tissue under a microscope, searching for visual signs that reveal whether cancer is present and, if so, what type and stage it has reached.
To a trained specialist, examining a pink, swirling tissue sample dotted with purple cells is like grading a test without a name on it — the slide contains vital information about the disease, but it offers no clues about who the patient is.
When AI Sees More Than Expected
That assumption does not fully apply to artificial intelligence systems now entering pathology labs. A new study led by researchers at Harvard Medical School shows that pathology AI models can infer demographic details directly from tissue slides. This unexpected ability can introduce bias into cancer diagnosis across different patient groups.
After evaluating several widely used AI models designed to identify cancer, the researchers found that these systems did not perform equally for all patients. Diagnostic accuracy varied based on patients’ self-reported race, gender, and age. The team also uncovered several reasons why these disparities occur.
To address the issue, the researchers developed a framework called FAIR-Path, which significantly reduced bias in the tested models.
“Reading demographics from a pathology slide is thought of as a ‘mission impossible’ for a human pathologist, so the bias in pathology AI was a surprise to us,” said senior author Kun-Hsing Yu, associate professor of biomedical informatics in the Blavatnik Institute at HMS and HMS assistant professor of pathology at Brigham and Women’s Hospital.
Yu emphasized that recognizing and correcting bias in medical AI is critical, since it can directly influence diagnostic accuracy and patient outcomes. The success of FAIR-Path suggests that improving fairness in cancer pathology AI, and possibly other medical AI tools, may not require major changes to existing systems.
The work, which was supported in part by federal funding, is described Dec. 16 in Cell Reports Medicine.
Putting Cancer AI to the Test
Yu and his colleagues examined bias in four commonly used pathology AI models currently being developed for cancer diagnosis. These deep-learning systems were trained on large collections of labeled pathology slides, allowing them to learn biological patterns and apply that knowledge to new samples.
The team evaluated the models using a large, multi-institutional dataset that included pathology slides from 20 different types of cancer.
Across all four models, performance gaps consistently emerged. The AI systems were less accurate for certain demographic groups defined by race, gender, and age. For example, the models struggled to distinguish lung cancer subtypes in African American patients and in male patients. They also showed reduced accuracy when classifying breast cancer subtypes in younger patients. In addition, the models had difficulty detecting breast, renal, thyroid, and stomach cancers in some demographic groups. Overall, these disparities appeared in roughly 29 percent of the diagnostic tasks analyzed.
According to Yu, these errors arise because the AI systems extract demographic information from the tissue images — and then rely on patterns linked to those demographics when making diagnostic decisions.
The findings were unexpected. “Because we would expect pathology evaluation to be objective,” Yu said. “When evaluating images, we don’t necessarily need to know a patient’s demographics to make a diagnosis.”
This led the researchers to ask a key question: Why was pathology AI failing to meet the same standard of objectivity?
Why Bias Appears in Pathology AI
The team identified three main contributors to the bias.
First, training data are often uneven. Tissue samples are easier to obtain from some demographic groups than others, resulting in imbalanced datasets. This makes it harder for AI models to accurately diagnose cancers in groups that are underrepresented, including some populations defined by race, age, or gender.
However, Yu noted that “the problem turned out to be much deeper than that.” In several cases, the models performed worse for certain demographic groups even when sample sizes were similar.
Further analysis pointed to differences in disease incidence. Some cancers occur more frequently in specific populations, allowing AI models to become especially accurate for those groups. As a result, the same models may struggle to diagnose cancers in populations where those diseases are less common.
The researchers also found that AI models can detect subtle molecular differences across demographic groups. For example, the systems may identify mutations in cancer driver genes and use them as shortcuts to classify cancer type — which can reduce accuracy in populations where those mutations are less prevalent.
“We found that because AI is so powerful, it can differentiate many obscure biological signals that cannot be detected by standard human evaluation,” Yu said.
Over time, this can cause AI models to focus on signals tied more closely to demographics than to the disease itself, weakening diagnostic performance across diverse patient groups.
Taken together, Yu said, these findings show that bias in pathology AI is influenced not only by the quality and balance of training data, but also by the way the models are trained to interpret what they see.
A New Approach to Reducing Bias
After identifying the sources of bias, the researchers set out to correct them.
They developed FAIR-Path, a framework based on an existing machine-learning method known as contrastive learning. This approach modifies AI training so that models focus more strongly on critical distinctions, such as differences between cancer types, while reducing attention to less relevant differences, including demographic characteristics.
When FAIR-Path was applied to the tested models, diagnostic disparities dropped by about 88 percent.
“We show that by making this small adjustment, the models can learn robust features that make them more generalizable and fairer across different populations,” Yu said.
The result is encouraging, he added, because it suggests that meaningful reductions in bias are possible even without perfectly balanced or fully representative training datasets.
Looking ahead, Yu and his team are working with institutions worldwide to study pathology AI bias in regions with different demographics, clinical practices, and laboratory settings. They are also exploring how FAIR-Path could be adapted for situations with limited data. Another area of interest is understanding how AI-driven bias contributes to broader disparities in health care and patient outcomes.
Ultimately, Yu said, the goal is to develop pathology AI systems that support human experts by delivering fast, accurate, and fair diagnoses for all patients.
“I think there’s hope that if we are more aware of and careful about how we design AI systems, we can build models that perform well in every population,” he said.
Authorship, funding, disclosures
Additional authors on the study include Shih-Yen Lin, Pei-Chen Tsai, Fang-Yi Su, Chun-Yen Chen, Fuchen Li, Junhan Zhao, Yuk Yeung Ho, Tsung-Lu Michael Lee, Elizabeth Healey, Po-Jen Lin, Ting-Wan Kao, Dmytro Vremenko, Thomas Roetzer-Pejrimovsky, Lynette Sholl, Deborah Dillon, Nancy U. Lin, David Meredith, Keith L. Ligon, Ying-Chun Lo, Nipon Chaisuriya, David J. Cook, Adelheid Woehrer, Jeffrey Meyerhardt, Shuji Ogino, MacLean P. Nasrallah, Jeffrey A. Golden, Sabina Signoretti, and Jung-Hsien Chiang.
Funding was provided by the National Institute of General Medical Sciences and the National Heart, Lung, and Blood Institute at the National Institutes of Health (grants R35GM142879, R01HL174679), the Department of Defense (Peer Reviewed Cancer Research Program Career Development Award HT9425-231-0523), the American Cancer Society (Research Scholar Grant RSG-24-1253761-01-ESED), a Google Research Scholar Award, a Harvard Medical School Dean’s Innovation Award, the National Science and Technology Council of Taiwan (grants NSTC 113-2917-I-006-009, 112-2634-F-006-003, 113-2321-B-006-023, 114-2917-I-006-016), and a doctoral student scholarship from the Xin Miao Education Foundation.
Ligon was a consultant of Travera, Bristol Myers Squibb, Servier, IntegraGen, L.E.K. Consulting, and Blaze Bioscience; received equity from Travera; and has research funding from Bristol Myers Squibb and Lilly. Vremenko is a cofounder and shareholder of Vectorly.
The authors prepared the initial manuscript and used ChatGPT to edit selected sections to improve readability. After using this tool, the authors reviewed and edited the content as needed and take full responsibility for the content of the published article.