Karen DeSalvo
Dr. Karen DeSalvo is a physician executive working at the intersection of medicine, public health, and information technology to help everyone, everywhere, live a healthier life. She leads a team of experts at Google who build helpful products, develop AI solutions focused on some of the biggest health challenges and bring information and insights to consumers, caregivers and communities with the aim of democratizing access to health and healthcare. She provides clinical leadership for Google employee health, including as part of the company COVID response team. Prior to joining Google, Dr. DeSalvo was National Coordinator for Health Information Technology and Assistant Secretary for Health (Acting) in the Obama Administration. Dr. DeSalvo served as the New Orleans Health Commissioner following Hurricane Katrina and was previously Vice Dean for Community Affairs and Health Policy at the Tulane School of Medicine where she was a practicing internal medicine physician, educator, and researcher. She is co-founder of the National Alliance to Impact the Social Determinants of Health. Dr. DeSalvo serves on the Council of the National Academy of Medicine and the Board of Directors for Welltower.
Research Areas
Authored Publications
Sort By
Preview abstract
Importance: Interest in artificial intelligence (AI) has reached an all-time high, and health care leaders across the ecosystem are faced with questions about where, when, and how to deploy AI and how to understand its risks, problems, and possibilities.
Observations: While AI as a concept has existed since the 1950s, all AI is not the same. Capabilities and risks of various kinds of AI differ markedly, and on examination 3 epochs of AI emerge. AI 1.0 includes symbolic AI, which attempts to encode human knowledge into computational rules, as well as probabilistic models. The era of AI 2.0 began with deep learning, in which models learn from examples labeled with ground truth. This era brought about many advances both in people’s daily lives and in health care. Deep learning models are task-specific, meaning they do one thing at a time, and they primarily focus on classification and prediction. AI 3.0 is the era of foundation models and generative AI. Models in AI 3.0 have fundamentally new (and potentially transformative) capabilities, as well as new kinds of risks, such as hallucinations. These models can do many different kinds of tasks without being retrained on a new dataset. For example, a simple text instruction will change the model’s behavior. Prompts such as “Write this note for a specialist consultant” and “Write this note for the patient’s mother” will produce markedly different content.
Conclusions and Relevance: Foundation models and generative AI represent a major revolution in AI’s capabilities, ffering tremendous potential to improve care. Health care leaders are making decisions about AI today. While any heuristic omits details and loses nuance, the framework of AI 1.0, 2.0, and 3.0 may be helpful to decision-makers because each epoch has fundamentally different capabilities and risks.
View details
Preview abstract
Technical innovations over the past 20 years have changed the way we live many parts of our daily lives, but there is resounding agreement that progress in health care has not kept pace. While many believe that technology will improve health outcomes, there is a real and persistent concern that technology companies do not understand the complexity of health and health care. This challenge is usually discussed as an either/or problem. Either technology companies must disrupt the way that health care works, or they won’t succeed because they will never understand the real world of health and health care. The authors believe that there is a third way — one that establishes a robust, thriving clinical team within a major technology company that brings a deep understanding of the current health care system to bear and a passion to make real improvements. However, clinical teams represent new functions for technology companies, and so they also represent a cultural shift. This article summarizes several years of experience building Google’s clinical team, and later adapting it during Covid-19, to offer six lessons for organizations embarking on similar journeys.
View details
Race- and Ethnicity-Stratified Analysis of an Artificial Intelligence–Based Tool for Skin Condition Diagnosis by Primary Care Physicians and Nurse Practitioners
David Way
Vishakha Gupta
Yi Gao
Guilherme De Oliveira Marinho
Jay David Hartford
Kimberly Kanada
Clara Eng
Kunal Nagpal
Lily Hao Yi Peng
Carter Dunn
Susan Jen Huang
Peggy Bui
(2022)
Preview abstract
Background:
Many dermatologic cases are first evaluated by primary care physicians or nurse practitioners.
Objective:
This study aimed to evaluate an artificial intelligence (AI)-based tool that assists with interpreting dermatologic conditions.
Methods:
We developed an AI-based tool and conducted a randomized multi-reader, multi-case study (20 primary care physicians, 20 nurse practitioners, and 1047 retrospective teledermatology cases) to evaluate its utility. Cases were enriched and comprised 120 skin conditions. Readers were recruited to optimize for geographical diversity; the primary care physicians practiced across 12 states (2-32 years of experience, mean 11.3 years), and the nurse practitioners practiced across 9 states (2-34 years of experience, mean 13.1 years). To avoid memory effects from incomplete washout, each case was read once by each clinician either with or without AI assistance, with the assignment randomized. The primary analyses evaluated the top-1 agreement, defined as the agreement rate of the clinicians’ primary diagnosis with the reference diagnoses provided by a panel of dermatologists (per case: 3 dermatologists from a pool of 12, practicing across 8 states, with 5-13 years of experience, mean 7.2 years of experience). We additionally conducted subgroup analyses stratified by cases’ self-reported race and ethnicity and measured the performance spread: the maximum performance subtracted by the minimum across subgroups.
Results:
The AI’s standalone top-1 agreement was 63%, and AI assistance was significantly associated with higher agreement with reference diagnoses. For primary care physicians, the increase in diagnostic agreement was 10% (P<.001), from 48% to 58%; for nurse practitioners, the increase was 12% (P<.001), from 46% to 58%. When stratified by cases’ self-reported race or ethnicity, the AI’s performance was 59%-62% for Asian, Native Hawaiian, Pacific Islander, other, and Hispanic or Latinx individuals and 67% for both Black or African American and White subgroups. For the clinicians, AI assistance–associated improvements across subgroups were in the range of 8%-12% for primary care physicians and 8%-15% for nurse practitioners. The performance spread across subgroups was 5.3% unassisted vs 6.6% assisted for primary care physicians and 5.2% unassisted vs 6.0% assisted for nurse practitioners. In both unassisted and AI-assisted modalities, and for both primary care physicians and nurse practitioners, the subgroup with the highest performance on average was Black or African American individuals, though the differences with other subgroups were small and had overlapping 95% CIs.
Conclusions:
AI assistance was associated with significantly improved diagnostic agreement with dermatologists. Across race and ethnicity subgroups, for both primary care physicians and nurse practitioners, the effect of AI assistance remained high at 8%-15%, and the performance spread was similar at 5%-7%.
View details
Development and Assessment of an Artificial Intelligence–Based Tool for Skin Condition Diagnosis by Primary Care Physicians and Nurse Practitioners in Teledermatology Practices
David Way
Vishakha Gupta
Yi Gao
Guilherme De Oliveira Marinho
Jay David Hartford
Kimberly Kanada
Clara Eng
Kunal Nagpal
Lily Hao Yi Peng
Carter Dunn
Susan Jen Huang
Peggy Bui
JAMA Network Open (2021)
Preview abstract
Importance: Most dermatologic cases are initially evaluated by nondermatologists such as primary care physicians (PCPs) or nurse practitioners (NPs).
Objective: To evaluate an artificial intelligence (AI)–based tool that assists with diagnoses of dermatologic conditions.
Design, Setting, and Participants: This multiple-reader, multiple-case diagnostic study developed an AI-based tool and evaluated its utility. Primary care physicians and NPs retrospectively reviewed an enriched set of cases representing 120 different skin conditions. Randomization was used to ensure each clinician reviewed each case either with or without AI assistance; each clinician alternated between batches of 50 cases in each modality. The reviews occurred from February 21 to April 28, 2020. Data were analyzed from May 26, 2020, to January 27, 2021.
Exposures: An AI-based assistive tool for interpreting clinical images and associated medical history.
Main Outcomes and Measures: The primary analysis evaluated agreement with reference diagnoses provided by a panel of 3 dermatologists for PCPs and NPs. Secondary analyses included diagnostic accuracy for biopsy-confirmed cases, biopsy and referral rates, review time, and diagnostic confidence.
Results: Forty board-certified clinicians, including 20 PCPs (14 women [70.0%]; mean experience, 11.3 [range, 2-32] years) and 20 NPs (18 women [90.0%]; mean experience, 13.1 [range, 2-34] years) reviewed 1048 retrospective cases (672 female [64.2%]; median age, 43 [interquartile range, 30-56] years; 41 920 total reviews) from a teledermatology practice serving 11 sites and provided 0 to 5 differential diagnoses per case (mean [SD], 1.6 [0.7]). The PCPs were located across 12 states, and the NPs practiced in primary care without physician supervision across 9 states. The NPs had a mean of 13.1 (range, 2-34) years of experience and practiced in primary care without physician supervision across 9 states. Artificial intelligence assistance was significantly associated with higher agreement with reference diagnoses. For PCPs, the increase in diagnostic agreement was 10% (95% CI, 8%-11%; P < .001), from 48% to 58%; for NPs, the increase was 12% (95% CI, 10%-14%; P < .001), from 46% to 58%. In secondary analyses, agreement with biopsy-obtained diagnosis categories of maglignant, precancerous, or benign increased by 3% (95% CI, −1% to 7%) for PCPs and by 8% (95% CI, 3%-13%) for NPs. Rates of desire for biopsies decreased by 1% (95% CI, 0-3%) for PCPs and 2% (95% CI, 1%-3%) for NPs; the rate of desire for referrals decreased by 3% (95% CI, 1%-4%) for PCPs and NPs. Diagnostic agreement on cases not indicated for a dermatologist referral increased by 10% (95% CI, 8%-12%) for PCPs and 12% (95% CI, 10%-14%) for NPs, and median review time increased slightly by 5 (95% CI, 0-8) seconds for PCPs and 7 (95% CI, 5-10) seconds for NPs per case.
Conclusions and Relevance: Artificial intelligence assistance was associated with improved diagnoses by PCPs and NPs for 1 in every 8 to 10 cases, indicating potential for improving the quality of dermatologic care.
View details
Preview abstract
Barriers to timely data collection and exchange hindered health departments throughout COVID-19, from fax machines creating bottlenecks for disease monitoring to inconsistent reporting of race and ethnicity. Modernizing public health data systems has become a bipartisan postpandemic imperative, with President Trump engaging the US Digital Service to improve data exchange and President Biden issuing an Executive Order on his second day in office to advance public health data and analytics.
These initiatives should be informed by the experience of digitizing health care delivery. The Health Information Technology for Economic and Clinical Health (HITECH) Act drove the near-universal adoption of certified electronic health records (EHRs). However, progress was not without pitfalls, from regulatory requirements affecting EHR usability, to new reporting, billing, and patient engagement processes disrupting workflows, to proprietary standards hindering interoperability.1 This Viewpoint explores lessons from HITECH for public health data modernization for COVID-19 and beyond.
View details
