Empowering Electronic Health Records Research with AI tools Requires More Statistical Thinking

Tianxi Cai, ScD
Harvard T.H. Chan School of Public Health

Harnessing the power of artificial intelligence (AI) in electronic health records (EHR) research unlocks transformative opportunities for translational science, particularly in the realms of precision medicine and optimizing healthcare delivery. AI tools—especially large language models (LLMs)—have demonstrated remarkable capabilities in extracting, summarizing, and predicting clinical information from the vast and complex data within EHRs, particularly from unstructured clinical notes. However, to fully realize the promise of AI in this context, a foundational integration of statistical thinking is indispensable.

Despite their utility, AI models often produce results that lack robustness and transparency. Issues such as hallucinated outputs, limited generalizability across healthcare systems, and insufficient quantification of uncertainty undermine their reliability. These limitations are exacerbated by real-world data challenges: missing or misclassified information, underrepresentation of rare diseases and small subgroups, unmeasured confounding in observational analyses, difficulties in transporting models across different patient populations, and degradation over time.

Statistical methodologies are critical for identifying and mitigating these challenges. Techniques from causal inference can help distinguish correlation from causation in non-randomized settings, while methods for uncertainty quantification can provide confidence in model outputs. Robust transfer learning algorithms can improve transportability across systems and over temporal periods. Hybrid frameworks that combine AI algorithms with rigorous statistical validation, sensitivity analyses, and bias correction strategies can improve both the interpretability and trustworthiness of AI-driven findings. Another practical barrier is the substantial computational cost of training and deploying sophisticated AI models, which may limit scalability or accessibility. Therefore, efficient model design and evaluation—guided by statisticians—are essential to balance performance with feasibility.

As AI becomes increasingly embedded in EHR-based research and clinical workflows, statisticians must take an active leadership role. Their expertise is crucial not only in developing robust analytical pipelines but also in ensuring that AI-driven insights are clinically valid, ethically sound, and ultimately actionable in real-world healthcare settings.

Biography
Tianxi Cai is the John Rock Professor of Population and Translational Data Science at the Harvard T.H. Chan School of Public Health and a Professor of Biomedical Informatics at Harvard Medical School. She co-directs the VERITY Bioinformatics Core at Brigham and Women’s Hospital and the Applied Bioinformatics Core at the Veterans Health Administration. As the founding director of the Translational Data Science Center for a Learning Health System and director of the Big Data Analytics Core at HMS, she leads efforts to provide statistical and biomedical informatics support to both the Harvard research community and external partners, including the VA and industry. Dr. Cai’s research focuses on developing innovative statistical and machine learning methods—such as semi-supervised learning, high-dimensional inference, robust transfer learning, graphical models, and federated learning—to address challenges in analyzing large-scale, multi-institutional biomedical data. Her team has successfully built scalable tools for extracting real-world evidence, performing text mining, and enabling predictive analytics using diverse sources including electronic health records, genomics, cohort studies, and clinical trials.

Gaining Confidence in Healthcare AI Requires More Than Confidence Intervals

Sachin Kheterpal, M.D., MBA
University of Michigan Medical School

Artificial intelligence and data analytics are now ubiquitous in everyday life, shaping how we search the internet, interact with customer service agents, and even watch competitive sports. With the explosion in both the volume and diversity of healthcare data, one might expect medicine to be poised for similar leaps forward in productivity and user experience through advanced analytics and statistical modeling.

However, healthcare is not just a “foundation model and validation” problem away from transformation. While data access, computing power, and statistical rigor are essential ingredients, they are not the key obstacles preventing lower costs or better clinical outcomes. Healthcare’s unique characteristics —strict data privacy standards, fear of medicolegal repercussions, idiosyncratic data interoperability, unmeasured patient and caregiver preferences, and clinician resistance to change—present substantial barriers that analytics alone cannot overcome.

In this talk, I will explore the features of clinical care that challenge the impact of statistical thinking in healthcare. We’ll discuss issues that cannot be addressed by analytic advances alone and identify potential solutions. Understanding these hurdles is crucial for anyone hoping to realize the full potential of AI in healthcare.

Biography
Sachin Kheterpal, M.D., MBA, is the Robert B Sweet Professor of Anesthesiology and Chair of the Department of Anesthesiology at the University of Michigan Medical School. Dr. Kheterpal’s career centers on leveraging information technology to enhance patient care, quality improvement, research, and education. Before entering academia, he cofounded a national Electronic Health Record software company which was acquired by General Electric, and then served as GE’s Global General Manager of Product Marketing, Development and Strategy for Clinical Information Systems.

He is the founder and Executive Director of the Multicenter Perioperative Outcomes Group (MPOG), a consortium of more than 70 health systems and 150 hospitals across the U.S., Canada, Middle East, and Europe that use a registry of 30 million patients for quality improvement, observational research, prospective clinical trials, and education. Outside the field of anesthesiology, Kheterpal is the Medical School’s Associate Dean for Research Information Technology and served as the founding co-director of U-M’s Precision Health Initiative. He has served on President Obama’s Precision Medicine Initiative Working Group, the NIH All of Us Advisory Panel, NIH Council of Councils, and the NIH Novel and Exceptional Technology and Research Advisory Committee (NExTRAC). He has published more than 200 peer-reviewed articles and served as principal investigator (PI) or co-PI of more than $75 million of sponsored grants and contracts.

Dr. Kheterpal continues to actively practice as a transplant anesthesiologist and serves as the PI for several multicenter clinical trials. In recognition of his contributions to the field of anesthesiology, he was elected to the National Academy of Medicine in 2022.