Chirag Nagpal
I am a Research Scientist with the Context in AI Research (CAIR) Team within Google Responsible AI. My interests are in building machine learning algorithms, tools, pipelines and software that augment decision making by being robust, fair and having broad practical utility across all domains of interest to Alphabet.
I received my PhD from the School of Computer Science at Carnegie Mellon, where I gained extensive experience in probabilistic inference and numerical optimization for semi-parametric graphical models with applications in patient risk stratification and treatment benefit assessment.
My graduate research has seen application in numerous patient risk prediction problems across multiple areas of healthcare [1] [2], has spun open source software [3] [4], and has been taught at graduate courses in MIT and Harvard [5].
I received my PhD from the School of Computer Science at Carnegie Mellon, where I gained extensive experience in probabilistic inference and numerical optimization for semi-parametric graphical models with applications in patient risk stratification and treatment benefit assessment.
My graduate research has seen application in numerous patient risk prediction problems across multiple areas of healthcare [1] [2], has spun open source software [3] [4], and has been taught at graduate courses in MIT and Harvard [5].
Research Areas
Authored Publications
Google Publications
Other Publications
Sort By
A Toolbox for Surfacing Health Equity Harms and Biases in Large Language Models
Heather Cole-Lewis
Nenad Tomašev
Liam McCoy
Leo Anthony Celi
Alanna Walton
Akeiylah DeWitt
Philip Mansfield
Sushant Prakash
Joelle Barral
Ivor Horn
Karan Singhal
Arxiv (2024) (to appear)
Preview abstract
Large language models (LLMs) hold immense promise to serve complex health information needs but also have the potential to introduce harm and exacerbate health disparities. Reliably evaluating equity-related model failures is a critical step toward developing systems that promote health equity. In this work, we present resources and methodologies for surfacing biases with potential to precipitate equity-related harms in long-form, LLM-generated answers to medical questions and then conduct an empirical case study with Med-PaLM 2, resulting in the largest human evaluation study in this area to date. Our contributions include a multifactorial framework for human assessment of LLM-generated answers for biases, and EquityMedQA, a collection of seven newly-released datasets comprising both manually-curated and LLM-generated questions enriched for adversarial queries. Both our human assessment framework and dataset design process are grounded in an iterative participatory approach and review of possible biases in Med-PaLM 2 answers to adversarial queries. Through our empirical study, we find that the use of a collection of datasets curated through a variety of methodologies, coupled with a thorough evaluation protocol that leverages multiple assessment rubric designs and diverse rater groups, surfaces biases that may be missed via narrower evaluation approaches. Our experience underscores the importance of using diverse assessment methodologies and involving raters of varying backgrounds and expertise. We emphasize that while our framework can identify specific forms of bias, it is not sufficient to holistically assess whether the deployment of an AI system promotes equitable health outcomes. We hope the broader community leverages and builds on these tools and methods towards realizing a shared goal of LLMs that promote accessible and equitable healthcare for all.
View details
Deep Cox Mixtures for Survival Regression
Proceedings of the 6th Machine Learning for Healthcare Conference, PMLR (2021), pp. 674-708
Preview abstract
Survival analysis is a challenging variation of regression modeling because of the presence of censoring, where the outcome measurement is only partially known, due to, for example, loss to follow up. Such problems come up frequently in medical applications, making survival analysis a key endeavor in biostatistics and machine learning for healthcare, with Cox regression models being amongst the most commonly employed models. We describe a new approach for survival analysis regression models, based on learning mixtures of Cox regressions to model individual survival distributions. We propose an approximation to the Expectation Maximization algorithm for this model that does hard assignments to mixture groups to make optimization efficient. In each group assignment, we fit the hazard ratios within each group using deep neural networks, and the baseline hazard for each mixture component non-parametrically.
We perform experiments on multiple real world datasets, and look at the mortality rates of patients across ethnicity and gender. We emphasize the importance of calibration in healthcare settings and demonstrate that our approach outperforms classical and modern survival analysis baselines, both in terms of discriminative performance and calibration, with large gains in performance on the minority demographics.
View details
No Results Found
