AI in Healthcare Part V: The Complex Path to Health Equity in the Age of AI
This is a five-part series examining the unmitigated risks of AI in healthcare
Part I: When Algorithms Prescribe Prejudice
Part II: A Ghost in the Machine
Part III: Can AI Break Free from Healthcare's History of Bias?
Part IV: Human Health and the AI Arms Race
Each of us inhabits multiple identities and because of that, any pursuit of equity must be intersectional.
It’s also why there will never be a direct path to health equity.
“I like to explain it as if you grew up in a village; you're not an expert in everything, and you have to rely on each other to help,” said Judy Gichoya, the Emory University radiologist who led the chest x-ray study.
Gichoya is Kenyan and points out that discrimination in the U.S. is often race-based, whereas in African countries people think in terms of clans.
“I cannot go to a specific country and say this is what you should be using (as an Artificial Intelligence tool to promote health equity)," she said. "You have to say this is a group that we've traditionally omitted and then figure out how to study it.”
Gichoya suggests implementing what she’s dubbed a “hive” learning approach to AI, meaning hubs of data scientists representing different communities and cultures, consistently coming together to share learnings, identify biases, and work to eliminate them. Gichoya helps host an event called “Health AI Bias Datathons,” which take place around the globe and create the hive learning environment she described. The event aims to "Shine a light on health disparities in chest imaging and mammography, creating a more equitable future for all." This year's Datathon was hosted in Atlanta, in August.
"If you are a white male, you are at the top of the hierarchy list, and if you transition to a white (transgender) female, you’re pretty much at the bottom of the list. It’s the same person, but we look at fairness in a very static way" – Judy Gichoya, MD of Emory University
“Those hive learnings and the data points tend to spark rapid learning, rapid experimentation, and rapid translation of health equity,” Gichoya said.
The learning is done with the understanding that “equity” and “fairness” have different meanings depending on one’s culture and are constantly evolving.
“And the reason is, we all live at an intersection,” she said. “Probably the most dramatic example I can give you is if you are a white male, you are at the top of the hierarchy list, and if you transition to a white (transgender) female, you’re pretty much at the bottom of the list. It’s the same person, but we look at fairness in a very static way, which makes it very difficult to translate and have meaningful conversations.”
“Could we get to a point where a sick person comes in and the operating table loses a leg, and the person still comes out fine and the operation went okay?” – Marzyeh Ghassemi, MIT AI in Healthcare researcher
Marzyeh Ghassemi, the MIT researcher, believes a more equitable approach would be to improve the safety standards by which the medical industry is regulated. She uses the metaphor of the aviation industry and the Federal Aviation Administration's regulations to protect against bad outcomes on an airplane – where people’s lives are also extremely vulnerable. Ghassemi points out that even with airlines and manufacturers (such as Boeing), grabbing headlines for safety failures, fatalities are extremely rare in aviation.
Emory Health AI Bias Datathon
In 2023 the risk of dying in an airplane crash was 0.03%. The five-year average from 2019-23 is still just 0.11%. Contrastingly, the risk of dying within 30 days of surgery hovers around 2%. The risk of dying from surgery varies wildly depending on the type of surgery being performed. The risk of fatality also varies wildly by race. As an example, for high-risk surgeries, Hispanics were 21% more likely to die within 30 days of a high-risk surgery than white patients, according to a study released in 2023; while Black patients’ risk of dying was 42% higher than that of whites.
“(The airlines) have prioritized safety to an extent where an airplane door fell off from an airplane at like 30,000 feet; the plane landed and everybody was okay,” Ghassemi said, referring to the Alaska Airlines’ Boeing 737 Max flight that took off from Portland, OR in January of 2024. “We are in a situation (in healthcare) where the plane is fine and the pilot accidentally killed somebody – but just one person in the airplane and it seems to usually be a woman or a minority. Could we get to a point where a sick person comes in and the operating table loses a leg, and the person still comes out fine and the operation went okay?”
While the threat of AI reinscribing existing health disparities is real, there’s also immense upside to improving equity and peeling back bias. How to create an apparatus that ensures a rule book all must play by is the challenge that keeps both of these researchers up at night.
“First, we have to be very inclusive in whose voices are at the table,” Gichoya said. “Today, if you look at the AI safety board in the U.S., it’s similar to a pharma board with pharma CEOs approving things. There are many, many people who have thought about the dangers of LLMs and were fired.”
“Second, we have to live in the gray because we don’t know enough. In these communities of learning, for AI to be successful, it will need to be a locally adaptable platform because those people will know how to treat the patient.”
As for the basketball documentary Netflix suggested, I think I’ll go ahead and watch it.
Stephen Norris is a strategic provider partnerships and management expert with a track record of driving growth and profitability. He has extensive experience building and expanding provider partnerships within the healthcare industry. Norris is skilled in contract negotiation, stakeholder management, and data analysis with a demonstrated ability to lead and motivate teams to deliver exceptional results. He has a deep understanding of the healthcare landscape and a passion for health equity through improving patient outcomes. He is #OpentoWork.
