Akshay Goel
Akshay Goel is a Physician Scientist and Software Engineer (Machine Learning) at Google Research, where he applies generative AI technologies to advance healthcare. His work focuses on medical information extraction, structured data optimization, and clinical decision support systems. Akshay holds a B.S. in Computer Science from Carnegie Mellon University and an M.D. from Rutgers University. He completed his residency in Diagnostic Radiology at NewYork-Presbyterian Hospital/Columbia University Medical Center and pursued further specialization through a fellowship in Abdominal Imaging and MRI at Weill Cornell Medicine.
Research Areas
Authored Publications
Sort By
Towards expert-level medical AI for real-time video consultations
Mahvish Nagda
Jihyeon Lee
Matthew Thompson
CJ Park
Tim Strother
Roma Ruparel
Teya Bergamaschi
Suhana Bedi
Meet Shah
Pavel Dubov
Toshiyuki Fukuzawa
Sam Schmidgall
Craig Schiff
Joseph Xu
Aliya Rysbek
Yana Lunts
Jan Freyberg
Rebecca Hemenway
David Racz
Carey Radebaugh
Joelle Barral
Kavi Goel
Kat Chou
James Manyika
Gregory Wayne
Yun Liu
Ethan Goh
Christina Chen
Ryutaro Tanno
arXiv (2026)
Preview abstract
Audio-visual interaction is the standard for patient-physician consultations, enabling natural communication and effective assessment of illness through non-verbal cues. While text-based AI has shown promise, it discards essential perceptual dimensions and limits patients who cannot articulate symptoms in writing. Early efforts to extend medical AI to audio-visual interaction have demonstrated feasibility, but not reached clinician-level performance. Here, we provide the first demonstration of expert-level AI in real-time clinical video consultations using AMIE (Articulate Medical Intelligence Explorer) in a video configuration. AMIE (Video) is a Gemini-based multi-agent system integrating low-latency dialogue, clinical reasoning, and real-time audio-visual perception. To guide development, we established a taxonomy and automated evaluations for clinical audio-visual cues in telehealth settings. In a randomized Objective Structured Clinical Examination (OSCE) study with 30 primary care physicians (PCPs), 15 patient actors and 100 clinical scenarios, we compared AMIE (Video), its text-only counterpart AMIE (Text), and PCPs consulting via video. Clinical evaluators rated AMIE (Video) on par or better than PCPs in history-taking, diagnosis, management, and physical observation and examination. Patient actors preferred AMIE's approach to assessing and explaining conditions, while PCPs were preferred for rapport and partnership building. In modality ablation, patient actors preferred AMIE (Video)'s interface over text chat for communicative effectiveness, convenience, and feeling understood. Limitations remain in fine anatomical precision, subtle affective nuances, and high-frequency movements. While further research is needed before real-world translation, these results mark an important milestone toward AI systems capable of augmenting care across the sensory complexity of clinical practice.
View details
ResidencyRL: Reinforcement Learning in Simulated Clinical Environments
Sam Schmidgall
Tim Strother
Alex Bijamov
CJ Park
Vahid Balazadeh
Min Woo Sun
Marius Guerard
Justin Chen
Vikram Dhillon
Kat Chou
Quoc Le
Raia Hadsell
Joelle Barral
Carey Radebaugh
Aleksandra Faust
David Racz
Lin Yang
Arxiv (2026)
Preview abstract
In medical education, physicians convert academic knowledge into clinical expertise through residency: years of training across thousands of encounters, with diverse sources of feedback and progressively greater autonomy. Much of clinical reasoning relies on the patient encounter, a dialogue in which a clinician elicits history, refines diagnostic hypotheses, and decides management under uncertainty. While large language models (LLMs) excel on static medical benchmarks, methods to optimize the full sequence of clinical decisions remain underdeveloped. We present ResidencyRL, a reinforcement learning (RL) method for training clinical artificial intelligence (AI) agents through simulated multi-turn clinical encounters (up to 60 dialogue turns and 8 tool calls per trajectory). ResidencyRL pairs the policy agent with LLM simulators capable of complex, adversarial behaviors, training against a structured reward aligned to diagnostic accuracy, management quality, communication, documentation, and safety. On held-out evaluations, the ResidencyRL agent improves diagnostic accuracy by 7.0% under adversarial conditions (88.0% vs. 81.0%) and reduces missed red flag rates by 31%, demonstrating rigorous mitigation of premature closure. Blinded expert clinicians validated these gains, preferring the trained agent in 87.6% of side-by-side comparisons. The procedural competencies transfer to unseen benchmarks: the agent outperforms the base model across all six clinical axes of the AMIE multi-visit benchmark, and shows consistent directional improvements on AgentClinic and CRAFT-MD. Our findings demonstrate that sequential clinical decision-making can be effectively learned through multi-turn RL in simulation, yielding robust, generalizable capabilities, paving the way towards clinical mastery. Prospective validation with real-world workflows remains necessary to establish clinical utility.
View details
Preview abstract
This study evaluates the proficiency of Large Language Models (LLMs) in accurately labeling clinical document excerpts. Our focus is on the assignment of potential or confirmed diagnoses and medical procedures to snippets of medical text sourced from unstructured clinical patient records. We explore how the performance of LLMs compare against human annotators in classifying these excerpts. Employing a few-shot, chain-of-thought prompting approach with the MIMIC-III dataset, Med-PaLM 2 showcases annotation accuracy comparable to human annotators, achieving a notable precision rate of approximately 92\% relative to the gold standard labels established by human experts.
View details
LLMs Accelerate Annotation for Medical Information Extraction
Almog Gueta
Omry Gilon
Chang Liu
Xiaohong Hao
Bolous Jaber
Shashir Reddy
Rupesh Kartha
Jean Steiner
Machine Learning for Health (ML4H), PMLR (2023)
Preview abstract
The unstructured nature of clinical notes within electronic health records often conceals vital patient-related information, making it challenging to access or interpret. To uncover this hidden information, specialized Natural Language Processing (NLP) models are required. However, training these models necessitates large amounts of labeled data, a process that is both time-consuming and costly when relying solely on human experts for annotation. In this paper, we propose an approach that combines Large Language Models (LLMs) with human expertise to create an efficient method for generating ground truth labels for medical text annotation. By utilizing LLMs in conjunction with human annotators, we significantly reduce the human annotation burden, enabling the rapid creation of labeled datasets. We rigorously evaluate our method on a medical information extraction task, demonstrating that our approach not only substantially cuts down on human intervention but also maintains high accuracy. The results highlight the potential of using LLMs to improve the utilization of unstructured clinical data, allowing for the swift deployment of tailored NLP solutions in healthcare.
View details
