D Shin
D Shin is a Senior Staff Software Engineer in the Augmented Reality (AR) team at Google. His research spans horizontally across the topics of input, sensors, audio, language, etc. for next-generation computing devices. He received his doctorate from MIT in 2016.
Research Areas
Authored Publications
Sort By
Soli-enabled Non-Contact Heart Rate Detection for Sleep and Meditation Tracking
Adam Bernstein
Alex Lee
Anupam Pathak
David Jorgensen
Eiji Hayashi
Haiguang Li
Ivan Poupyrev
Jaime Lien
Jian Cui
Jian Wang
Jihan Li
Jin Yamanaka
Lauren Bedal
Luzhou Xu
Nicholas Gillian
Qian Zhang
Rajeev Nongpiur
Shwetak Patel
Trausti Thormundsson
Nature Scientific Reports (2023)
Preview abstract
Heart rate (HR) is a crucial physiological signal that can be used to monitor health and fitness. Traditional methods for measuring HR require wearable devices, which can be inconvenient or uncomfortable, especially during sleep and meditation. Noncontact HR detection methods employing microwave radar can be a promising alternative. However, they usually use high-gain antennas and require the sensor to face the user’s chest or back, making them difficult to integrate into a portable device and unsuitable for sleep and meditation tracking applications. This study presents a novel approach for noncontact HR detection using a miniaturized Soli radar chip embedded in a Google Nest Hub. The chip has a 6.5 mm × 5 mm × 0.9 mm dimension and can be easily integrated into various devices. Our approach utilizes advanced signal processing and machine learning techniques to extract HRs from radar signals. The approach is validated on a sleep dataset (62 users, 498 hours) and a meditation dataset (114 users, 1131 minutes). The approach achieves a mean absolute error (MAE) of 1.69 bpm and a mean absolute percentage error (MAPE) of 2.67% on the sleep dataset. On the meditation dataset, the approach achieves an MAE of 1.05 bpm and a MAPE of 1.56%. The recall rates for the two datasets are 88.53% and 98.16%, respectively. This study represents the first application of the noncontact HR detection technology to sleep and meditation tracking, offering a promising alternative to wearable devices for HR monitoring during sleep and meditation.
View details
Sleep-wake Detection With a Contactless, Bedside Radar Sleep Sensing System
Michael Dixon
Logan Schneider
Jeffrey Yu
Jonathan Hsu
Anupam Pathak
Reena Singhal Lee
Mark Rajan Malhotra
Ken Mixter
Mike McConnell
James Taylor
Shwetak Patel
Google (2021)
Preview abstract
Sleep constitutes nearly ⅓ of the human lifespan, yet most individuals are unaware of precisely how much or how well they’re sleeping. With low-energy radar technology, integrated into the new second-generation Nest Hub device, users can access a contactless, bedside sleep-sensing system. Radar-based detection of sub-centimeter body movements enables the passive monitoring of sleep patterns with relative ease (e.g., no need to remember to charge the device or turn it on) and without the need for cameras, microphones, or physical contact with the user. Moreover, privacy-preserving, on-device processing of the raw sensor data is employed, so only the results of the algorithm (e.g., awake or asleep) are securely uploaded and provided to the user.
This paper provides a detailed understanding of the capabilities of the second-generation Nest Hub’s Sleep Sensing feature, including algorithm development and validation. In brief, the deep learning algorithm, when compared to gold-standard clinical sleep assessment, achieved overall epoch-by-epoch sleep-wake accuracy of 87% in healthy sleepers: correctly detecting 96% of sleep epochs and 55% of wake epochs. This accuracy is comparable to published results for other clinical- and consumer-grade devices.
View details
