Reprinted: Gastroenterology & Endoscopy News, 1-20-2025
Artificial intelligence has the potential to deliver unprecedented value in healthcare, but the same equity challenges that persist in other areas of healthcare must be addressed in the world of AI as well, noted Maia Hightower, MD, MPH, MBA, in a presentation at the American Society for Gastrointestinal Endoscopy’s 2024 Global Gastroenterology and Artificial Intelligence Summit.
AI has great potential to improve value, quality, cost, and the patient and care team experience, said Dr. Hightower, the CEO of Equality AI, a company that provides healthcare AI quality assurance, risk management, and compliance consulting and software. However, she added, “80% of AI projects in healthcare fail.”
Noting that one of the persistent barriers to the translation of AI into healthcare practice is AI bias and health inequity, Dr. Hightower said “bias occurs throughout the AI life cycle.”
Clinicians involved in using AI should know the points along the way at which bias can be introduced, she said. At the data creation level, for example, real-world data might not be representative of the target population. In addition, bias can be introduced in model development, such as labeling errors, and model evaluation often excludes analyses of subpopulations, she said. Decisions made during model deployment also may introduce biased decision-making, she added.
Dr. Hightower shared results of a study published in Science (2019;366[6464]:447-453) that examined bias in an AI algorithm used to manage the health of populations. In the algorithm’s original form, Black patients were 50% less likely than white patients to be referred for care, despite being equally sick. In the study, Ziad Obermeyer, MD, of the University of California, Berkeley, and his co-investigators revised the algorithm to mitigate bias, resulting in 44% of Black patients being referred for care compared with 56% of white patients.
Quality Assurance for AI
Potential solutions to the problem of inequity in AI include involving more diverse teams in the creation of models and improving AI governance by involving patients, clinicians, ethicists and other stakeholders. A responsible quality assurance and management platform for AI calls for consistent monitoring, auditing and validating, Dr. Hightower said.
Dr. Hightower encouraged clinicians to get involved in improving AI whenever possible to increase the relevance and accuracy of models. In addition, she said, “clinicians should provide oversight of AI systems … to ensure they are operating as intended and providing safe and effective recommendations.” However, it is equally important for clinicians to understand the limitations of AI systems to provide the best possible patient care and to speak up with concerns about any AI systems to colleagues, administrators and regulators, Dr. Hightower said. “While AI promises many benefits, clinicians play a critical role in ensuring these systems are used responsibly and safely.”
Promise and Perils
“AI has the potential to significantly improve equity in gastroenterology by addressing gaps in access to healthcare, early diagnosis and treatment outcomes across diverse populations,” Summit co-chair Prateek Sharma, MD, the president of the ASGE, told Gastroenterology & Endoscopy News.
“We are at a critical juncture, such that if the appropriate AI tools are used, we can have a major impact on improving access to care in underserved areas, personalizing risk assessment, reducing bias in medical decision-making, and using multilingual and culturally competent AI tools,” said Dr. Sharma, a professor of medicine at the University of Kansas School of Medicine, in Kansas City.
Despite this potential, Dr. Sharma added, challenges to the use of AI for improving equity in gastroenterology include bias in algorithms, limited data availability and quality in underserved areas, disparities in access to digital infrastructure, a lack of trust in AI, and concerns about patient privacy, consent and data ownership. Gastroenterologists should be committed to addressing these issues.