PhD Thesis Defense: Purity Mugambi, Identifying and Intervening on Treatment Disparities in Electronic Health Record Data
Content
Speaker:
Abstract:
Machine learning (ML) researchers have increasingly used electronic health record (EHR) datasets, especially those that are anonymized and publicly accessible, to train models that could be deployed in the real world. Simultaneously, clinical researchers have shown that systemic injustices creep into health systems leading to vast and pervasive disparities in treatment of patients based on factors such as sex, gender, race/ethnicity, and socioeconomic status. This produces substandard care for patients, but also, makes poor use of scarce healthcare resources and contributes to longstanding social injustice. Therefore, it is vital that creators of ML models for healthcare understand the prevalence of such disparities in EHR datasets, so they can make appropriate considerations when training models and interpreting their findings.
This thesis seeks to understand the extent of health inequity captured in EHR data and investigate how ML models can be redesigned to ensure they maintain high performance for the patient groups negatively aIected by those inequities. To that end, we build tools to; 1) quantify the disparities in treatment of patients across multiple datasets, 2) automate cohort extraction from large databases to reduce the time demand in multi-dataset analyses, and 3) optimize between personalization and generalization to generate cohort- specific models that can improve performance for underrepresented patient groups. This thesis, therefore, is structured in these three main parts.
First, to understand the prevalence of disparities in EHR datasets, we build a tool to run multiple hypothesis tests across multiple datasets to quantify diIerences in proportions of patients who received various treatments and in quantities of the treatment that they received. This tool also runs multiple regression analyses to compute the association between treatments and patient outcomes, which is vital in understanding the eIect of treatment disparities. This system helps answer questions about how disparities compare across datasets and how they have changed temporally.
Second, in light of the findings on existing treatment inequities, we develop methods for improving performance of predictive ML models on minoritized patient groups. We explore the tradeoI between personalization and generalization and develop hierarchical models that have higher predictive accuracy for smaller patient cohorts even in datasets with highly imbalanced labels.
Third, we investigate ways to automate cohort extraction from EHR databases, a task that is critical to many observational studies yet is currently manual and extremely time consuming. We propose a language-model-based schema matching algorithm and evaluate it using five small language models across two typical criteria. By matching columns across databases, the researcher(s) can run cohort selection criteria queries on multiple databases faster, saving them crucial time currently invested to understand schemas of databases they have previously not worked with.
This thesis makes contributions to the understanding of the pervasiveness of disparities in treatment (especially of acute myocardial infarction) and how that diIers across multiple datasets. Most importantly, through this work, we develop tools that allow clinical researchers to quickly search their private datasets for such disparities, and compare against insights learned from other datasets. These tools are easy to use and can be used for diIerent disease use cases. The findings obtained from using these tools empower clinical stakeholders to make informed decisions about their systems of care with the goal of closing existing equity gaps. They also inform ML researchers on existing inequities, requiring them to redesign these models such that eIects of the disparities present in data are reduced in downstream applications. Finally, this thesis shows one approach through which ML models can be redesigned to ensure they have high predictive performance for underrepresented patient subgroups who typically are negatively aIected by existing health inequities.
Advisor:
Ina Fiterau