John Hernandez, PhD, MPP
Director, Head of Clinical Research Center of Excellence and Health Impact team
Authored Publications
Sort By
Capturing Real-World Habitual Sleep Patterns with a Novel User-centric Algorithm to Pre-Process Fitbit Data in the All of Us Research Program: Retrospective observational longitudinal study
Hiral Master
Jeffrey Annis
Karla Gleichauf
Lide Han
Peyton Coleman
Kelsie Full
Neil Zheng
Doug Ruderfer
Logan Schneider
Evan Brittain
Journal of Medical Internet Research (2025)
Preview abstract
Background:
Commercial wearables such as Fitbit quantify sleep metrics using fixed calendar times as default measurement periods, which may not adequately account for individual variations in sleep patterns. To address this limitation, experts in sleep medicine and wearable technology developed a user-centric algorithm designed to more accurately reflect actual sleep behaviors and improve the validity of wearable-derived sleep metrics.
Objective:
This study aims to describe the development of a new user-centric algorithm, compare its performance with the default calendar-relative algorithm, and provide a practical guide for analyzing All of Us Fitbit sleep data on a cloud-based platform.
Methods:
The default and user-centric algorithms were implemented to preprocess and compute sleep metrics related to schedule, duration, and disturbances using high-resolution Fitbit sleep data from 8563 participants (median age 58.1 years, 6002/8341, 71.96%, female) in the All of Us Research Program (version 7 Controlled Tier). Variations in typical sleep patterns were calculated by examining the differences in the mean number of primary sleep logs classified by each algorithm. Linear mixed-effects models were used to compare differences in sleep metrics across quartiles of variation in typical sleep patterns.
Results:
Out of 8,452,630 total sleep logs collected over a median of 4.2 years of Fitbit monitoring, 401,777 (4.75%) nonprimary sleep logs identified by the default algorithm were reclassified as primary sleep by the user-centric algorithm. Variation in typical sleep patterns ranged from –0.08 to 1. Among participants with the greatest variation in typical sleep patterns, the user-centric algorithm identified significantly more total sleep time (by 17.6 minutes; P<.001), more wake after sleep onset (by 13.9 minutes; P<.001), and lower sleep efficiency (by 2.0%; P<.001), on average. Differences in sleep stage metrics between the 2 algorithms were modest.
Conclusions:
The user-centric algorithm captures the natural variability in sleep schedules, providing an alternative approach to preprocess and evaluate sleep metrics related to schedule, duration, and disturbances. A publicly available R package facilitates the implementation of this algorithm for clinical and translational research.
View details
A personal health large language model for sleep and fitness coaching
Anastasiya Belyaeva
Zhun Yang
Nick Furlotte
Chace Lee
Erik Schenck
Yojan Patel
Jian Cui
Logan Schneider
Robby Bryant
Ryan Gomes
Allen Jiang
Roy Lee
Javier Perez
Jamie Rogers
Cathy Speed
Shyam Tailor
Megan Walker
Jeffrey Yu
Tim Althoff
Conor Heneghan
Mark Malhotra
Leor Stern
Shwetak Patel
Shravya Shetty
Jiening Zhan
Daniel McDuff
Nature Medicine (2025)
Preview abstract
Although large language models (LLMs) show promise for clinical healthcare applications, their utility for personalized health monitoring using wearable device data remains underexplored. Here we introduce the Personal Health Large Language Model (PH-LLM), designed for applications in sleep and fitness. PH-LLM is a version of the Gemini LLM that was finetuned for text understanding and reasoning when applied to aggregated daily-resolution numerical sensor data. We created three benchmark datasets to assess multiple complementary aspects of sleep and fitness: expert domain knowledge, generation of personalized insights and recommendations and prediction of self-reported sleep quality from longitudinal data. PH-LLM achieved scores that exceeded a sample of human experts on multiple-choice examinations in sleep medicine (79% versus 76%) and fitness (88% versus 71%). In a comprehensive evaluation involving 857 real-world case studies, PH-LLM performed similarly to human experts for fitness-related tasks and improved over the base Gemini model in providing personalized sleep insights. Finally, PH-LLM effectively predicted self-reported sleep quality using a multimodal encoding of wearable sensor data, further demonstrating its ability to effectively contextualize wearable modalities. This work highlights the potential of LLMs to revolutionize personal health monitoring via tailored insights and predictions from wearable data and provides datasets, rubrics and benchmark performance to further accelerate personal health-related LLM research.
View details
Smartphone use in a large US adult population: Temporal associations between objective measures of usage and mental well-being
Ari Winbush
Benjamin Nelson
Nicholas Allen
Andrew Barakat
Daniel McDuff
Conor Heneghan
Allen Jiang
PNAS (2025)
Preview abstract
Smartphones are a vital tool for most people. They facilitate many everyday tasks and as a result they have become ubiquitous and indispensable. There are concerns about how the use of these devices may impact mental health and wellbeing. Yet, there are few studies that have reported objective data about phone usage from large and diverse cohorts and studies have found low correlations between subjective and objective smartphone use. In order to better elucidate these complex interactions, it is important to understand and characterize what resembles “normative” smartphone use behavior. In this paper, we present normative patterns of objectively measured phone usage from a large prospective observational study. We analyze a quarter of a million days of phone usage data from 10,099 adult subjects that provides objective longitudinal data over a four week period in the US general population. Contrary to popular belief, our model shows little support for the conclusion that smartphone use predicts mood the following week or that mood predicts smartphone use the following week, with some results differing depending on whether the effects are within-person or between-person. Lastly, while some findings are statistically significant, the effect sizes of these results are minimal, suggesting little to no impact in real-world settings and therefore a lack of clinical significance.
View details
The Anatomy of a Personal Health Agent
Ahmed Metwally
Ken Gu
Jiening Zhan
Kumar Ayush
Hong Yu
Akshay Paruchuri
Amy Lee
Qian He
Zhihan Zhang
Isaac Galatzer-Levy
Xavi Prieto
Andrew Barakat
Ben Graef
Yuzhe Yang
Daniel McDuff
Brent Winslow
Shwetak Patel
Girish Narayanswamy
Conor Heneghan
Max Xu
Jacqueline Shreibati
Mark Malhotra
Orson Xu
Tim Althoff
Tony Faranesh
Nova Hammerquist
Vidya Srinivas
arXiv (2025)
Preview abstract
Health is a fundamental pillar of human wellness, and the rapid advancements in large language models (LLMs) have driven the development of a new generation of health agents. However, the solution to fulfill diverse needs from individuals in daily non-clinical settings is underexplored. In this work, we aim to build a comprehensive personal health assistant that is able to reason about multimodal data from everyday consumer devices and personal health records. To understand end users’ needs when interacting with such an assistant, we conducted an in-depth analysis of query data from users, alongside qualitative insights from users and experts gathered through a user-centered design process. Based on these findings, we identified three major categories of consumer health needs, each of which is supported by a specialist subagent: (1) a data science agent that analyzes both personal and population-level time-series wearable and health record data to provide numerical health insights, (2) a health domain expert agent that integrates users’ health and contextual data to generate accurate, personalized insights based on medical and contextual user knowledge, and (3) a health coach agent that synthesizes data insights, drives multi-turn user interactions and interactive goal setting, guiding users using a specified psychological strategy and tracking users’ progress. Furthermore, we propose and develop a multi-agent framework, Personal Health Insight Agent Team (PHIAT), that enables dynamic, personalized interactions to address individual health needs. To evaluate these individual agents and the multi-agent system, we develop a set of N benchmark tasks and conduct both automated and human evaluations, involving 100’s of hours of evaluation from health experts, and 100’s of hours of evaluation from end-users. Our work establishes a strong foundation towards the vision of a personal health assistant accessible to everyone in the future and represents the most comprehensive evaluation of a consumer AI health agent to date.
View details
Analysis of objective and subjective sleep metrics and smartphone usage patterns
Conor Heneghan
Daniel McDuff
Ari Winbush
Nicholas Allen
Allen Jiang
Andrew Barakat
Logan Schneider
Benjamin Nelson
Ben Yetton
2024
Preview abstract
Analysis of objective and subjective sleep metrics and smartphone usage patterns
Conor Heneghan, , Daniel McDuff, Ari Winbush, Nicholas Allen, John Hernandez, Allen Jiang,, Andrew Barakat, Logan Schneider, Benjamin Nelson, Ben Yetton
Consumer Health Research Team, Google Inc.
Department of Psychology, University of Oregon
Verily Life Sciences
Department of Psychiatry, Harvard Medical School and Beth Israel Deaconess Medical Center
Introduction: The Digital Wellbeing Study is an IRB approved joint study between the University of Oregon and Google to investigate how smartphone usage interacts with objective and
subjective parameters of well-being such as sleep, exercise and stress. The study recruited a demographically diverse population who each wore a smartwatch and installed a smartphone app linked to the study. Participants completed demographic and health questionnaires including the PROMIS Sleep Disturbance (SD) Short Form. Aims of the study included (a) whether objective sleep duration was correlated with smartphone use, and (b) whether smartphone usage could predict the subjective self reported sleep instrument.
Methods: There was sufficient data from 7,499 users to conduct a population modeling analysis. An Ordinary Least Squares linear model was used as a predictor of each subject’s average total sleep time (TST) and their SD t-score. The inputs to the model included demographics, and population z-scored activity measures (steps, sedentary time, time driving, time at work, home and other locations, phone screen time, frequency of phone unlocks)
over seven days prior to the survey.
Results: The activity measures and baseline demographics could only explain a small amount of the overall variance in TST and SD (R^2=0.04 for TST and R^2=0.05 for SD). Phone screen
time was a statistically significant predictor of both TST (-8.19 mins, p< 0.001) and self-reported sleep disruption (0.611 t-score units, p< 0.001). The number of phone unlocks was a predictor of variability in TST (-3.33 mins, p< 0.001) suggesting that longer session times are correlated with greater TST variability. The effects are minimal (e.g., a subject who has one standard
deviation greater phone screen time than average would be predicted to only see a 2% reduction in TST, and a 0.6% increase in perceived sleep disturbance). Time driving and step count were
also minor predictors of SD and TST.
Conclusion: At a population level, average activity measures from wearables and smartphones such as steps, smartphone usage time, sedentary activity etc. are limited predictors of
objective sleep metrics such as Total Sleep Time, and subjective sleep metrics such as the PROMIS Sleep Disturbance t-score.
Support (if any): This research was funded by Google Inc.
View details
Evidence of Differences in Diurnal Electrodermal Patterns by Mental Health Status in Free-Living Data
Daniel McDuff
Isaac Galatzer-Levy
Andrew Barakat
Conor Heneghan
Samy Abdel-Ghaffar
Jake Sunshine
Lindsey Sunden
Allen Jiang
Ari Winbush
Benjamin Nelson
Nicholas Allen
medRxiv (2024)
Preview abstract
Electrodermal activity (EDA) is a standardized measure of sympathetic arousal that has been linked to depression in laboratory experiments. However, the inability to measure EDA passively over time and in the real-world has limited conclusions that can be drawn about EDA as an indicator of mental health status outside of a controlled setting. Recent smartwatches have begun to incorporate wrist-worn continuous EDA sensors that enable longitudinal measurement in every-day life. This work presents the first example of passively collected, diurnal variations in EDA present in people with depression, anxiety and perceived stress. Subjects who were depressed had higher tonic EDA and heart rate, despite not engaging in greater physical activity, compared to those that were not depressed. EDA measurements showed differences between groups that were most prominent during the early morning. We did not observe amplitude or phase differences in the diurnal patterns.
View details
What Are The Odds? Language Models are Capable of Probabilistic Reasoning
Akshay Paruchuri
Shun Liao
Jake Sunshine
Tim Althoff
Daniel McDuff
arXiv (2024)
Preview abstract
Language models (LM) are capable of remarkably complex linguistic tasks; however, numerical reasoning is an area in which they frequently struggle. An important but rarely evaluated form of reasoning is understanding probability distributions. In this paper we focus on evaluating the probabilistic reasoning capabilities of LMs using idealized and real-world statistical distributions. We perform a systematic evaluation of state-of-the-art LMs on three tasks: estimating percentiles, drawing samples, and calculating probabilities. We find that zero-shot performance varies dramatically across different families of distributions and that performance can be improved significantly by using anchoring examples (shots) from within a distribution, or to a lesser extent across distributions within the same family. For real-world distributions, the absence of in-context examples can be substituted with context from which the LM can retrieve some statistics. Finally, we show that simply providing the mean and standard deviation of real-world distributions improves performance. To conduct this work, we developed a comprehensive benchmark distribution dataset with associated question-answer pairs that we release publicly, including questions about population health, climate, and finance.
View details
Research Protocol for the Google Health Digital Wellbeing Study
Ari Winbush
Nicholas Allen
Andrew Barakat
Felicia Cordeiro
Daniel McDuff
Allen Jiang
Ryann Crowley
JMIR Research Protocols (2024)
Preview abstract
The impact of digital device use on health and wellbeing is a pressing question to which individuals, families, schools, policy makers, legislators, and digital designers are all demanding answers. However, the scientific literature on this topic to date is marred by small and/or unrepresentative samples, poor measurement of core constructs (e.g., device use, smartphone addiction), and a limited ability to address the psychological and behavioral mechanisms that may underlie the relationships between device use and wellbeing. A number of recent authoritative reviews have made urgent calls for future research projects to address these limitations. The critical role of research is to identify which patterns of use are associated with benefits versus risks, and who is more vulnerable to harmful versus beneficial outcomes, so that we can pursue evidence-based product design, education, and regulation aimed at maximizing benefits and minimizing risks of smartphones and other digital devices. We describe a protocol for a Digital Wellbeing Study (DWB) to help answer these questions.
View details
Preview abstract
This Op-ed is by leaders from the American Heart Association, Digital Medicine Society and Google involved in a Digital Medicine Society-sponsored project on digital measures for physical activity. The Op-ed summarizes evidence that the technology exists today to digitally measure physical activity in the broad population – and, by measuring it the right way, we can embrace it as the ‘6th vital sign’ and enter a new era of healthcare centered on proactive patient care.
View details
Predicting subjective sleep impairment and disturbance from wearable sleep data
Conor Heneghan
Ben Yetton
Daniel McDuff
Nicholas Allen
Andrew Barakat
Allen Jiang
Logan Schneider
Benjamin Nelson
Ari Winbush
2024
Preview abstract
Introduction:
Wearables offer a scalable, passive and objective measure of sleep health. However, prior reported correlations (spearman) between subjective and wearable derived sleep measures have been modest (rS=0.3-0.46). We set out to determine if wearables adequately capture subjective feelings of sleep disturbance and impairment in a large, diverse ecologically valid sleep study.
Methods:
Subject data (n=2922, mean age= 45.4 (12.6), 74% female) came from the Digital Wellbeing Study: a joint study between the University of Oregon and Google to investigate how smartphone usage impacts well-being. Wearable (Fitbit) derived sleep metrics were summarized across the week prior to the administration of the PROMIS Sleep Disturbance (SD) and Sleep Related Impairment (SR) Short Form surveys. A series of stepwise OLS regressions were used to test the predictive power of each sleep metric over a baseline model of age and sex.
Results:
Sleep variables of total sleep time, resting heart rate, and the variability in total sleep time and restlessness (accelerometer based metric) improved both SI and SD above a baseline model (SIBaseline adjR2=0.087, SDBaseline adjR2=0.024). Deep (e.g. N3) minutes uniquely improved SI model fit, while longest wake length and total wake minutes improved SD fit. REM percent and normalized nightly heart rate did not improve model fit. The final model explained 12.9% of the variance of SI, and 8.4% of the variance of SD. The most predictive single sleep metric was the variability in total sleep time (adjR2=0.104) for SI, and total sleep time for SD (age & sex included). Fitbit’s composite “Sleep Score” was the single best predictor of SD when included in analysis (age and sex excluded).
Conclusion: As demonstrated in previous studies, wearable derived sleep metrics are modest predictors of perceived sleep disturbance or sleep related impairment. Composite metrics that include measures of sleep variability are recommended.
Support: This research was funded by Google Inc.
View details
