By Sophia Kenward, SWHR Communications and Policy Intern
One year ago, SWHR published a blog on the promises and perils of artificial intelligence (AI) for women’s health. Since then, the debate has continued surrounding the potential benefits and harms of harnessing the power of AI in health care.
Last month, an investigation by STAT found that U.S. regulators approved more than 160 medical AI products in recent years without requiring manufacturers to publicly document their testing on patients of different genders, races, and geographies. Only 73 manufacturers disclosed the data used to validate device performance. Only 13 of those 73 provided a gender breakdown, and just seven reported race and ethnicity.
Why does this matter? Machine learning algorithms, a branch of AI, work by analyzing statistical patterns within datasets. Therefore, an AI product is only as effective and unbiased as the data that it is trained on. If a product is not appropriately trained on datasets from diverse populations, it may not work effectively for all patients and may even exacerbate existing biases in the health care system.
For example, an AI-based symptom checker app was criticized for giving a less serious diagnosis for a female patient reporting the same symptoms as a male patient. For a 60-year-old female smoker reporting sudden onset chest pain and nausea, the app suggested a panic attack or pain caused by inflammation. For a 60-year-old male smoker reporting the same symptoms, the app advised to visit the emergency room, suspecting a heart attack. Unfortunately, women experiencing heart attack symptoms are more likely to be misdiagnosed in our health care system, despite heart disease being the No. 1 cause of death among women.
In a letter published in JAMA late last year, researchers analyzed more than 2,600 peer-reviewed articles on deep learning algorithms across six medical disciplines. They reported the training data used were disproportionately from patient cohorts from California, Massachusetts, and New York, with little to no representation from the remaining 47 states. These states have economic, educational, social, behavioral, ethnic, and cultural features that are not representative of the nation as a whole. These social determinants of health matter when it comes to creating medical AI products, and data should reflect the features of the populations assessed or treated using AI algorithms.
Artificial intelligence products could be particularly transformative in diagnosing and treating breast cancer. Thanks to new developments in breast cancer AI products, high-quality decision-making could become more accessible and affordable to patients than ever before. AI may potentially improve evaluation of breast cancer screening mammograms, which could in turn improve prognosis and reduce mortality rates. AI could likely make a tremendous impact on the health and well-being of women, so long as developers mitigate biases in their datasets.
According to STAT’s report on FDA-approved medical AI, 10 products were cleared for use in breast imaging, but just one publicly disclosed the racial demographics of the dataset used to validate the tool. Breast cancer risk, presentation, and outcomes differ across patient populations. For instance, white women are slightly more likely to develop breast cancer, while Black women are more likely to develop more aggressive cancer, to develop cancer earlier in life, and to die from breast cancer.
Thus, it’s important to know whether these AI products have been trained and tested on diverse data. Including diverse populations in research — and publicly reporting that information — is imperative to ensuring that AI algorithms work for women and people of color and that patients feel they can trust these products.
Many AI products currently on the market will undoubtedly change the medical field, providing new and innovative solutions to existing health problems. AI has been especially useful during the COVID-19 pandemic. Given the infectious nature of the virus, AI products can be used to reduce human contact and protect health care workers from transmission. Some health care organizations have started to rely on AI to assist with the COVID-19 vaccine rollout to schedule vaccines, streamline patient communications, and even prioritize access.
The U.S. Food and Drug Administration (FDA) in January released an action plan for AI and machine learning that calls for greater transparency from AI manufacturers. The plan also encourages manufacturers to include diverse patient populations in trial datasets to reduce algorithm biases.
The FDA’s plan is a step in the right direction to ensure AI products work effectively and safely for women, people of color, and other members of diverse populations. But more needs to be done, including incentivizing diverse perspectives in the research and technology fields — both among study participants and researchers. It is imperative that AI developers and regulators consider race, ethnicity, and gender in their datasets to work toward improved health equity for women and people of color.