David Pearl
Authored Publications
Sort By
APG: Audioplethysmography for Cardiac Monitoring in Hearables
Rich Howard
Longfei Shangguan
Trausti Thormundsson
MobiCom 2023: The 29th Annual International Conference On Mobile Computing And Networking (MobiCom), Association for Computing Machinery (ACM) (to appear)
Preview abstract
This paper presents Audioplethysmography (APG), a novel cardiac monitoring modality for active noise cancellation (ANC) headphones. APG sends a low intensity ultrasound probing signal using an ANC headphone's speakers and receives the echoes via the on-board feedback microphones. We observed that, as the volume of ear canals slightly changes with blood vessel deformations, the heartbeats will modulate these ultrasound echoes. We built mathematical models to analyze the underlying physics and propose a multi-tone APG signal processing pipeline to derive the heart rate and heart rate variability in both constrained and unconstrained settings. APG enables robust monitoring of cardiac activities using mass-market ANC headphones in the presence of music playback and body motion such as running.
We conducted an eight-month field study with 153 participants to evaluate APG in various conditions. Our studies conform to the (Institutional Review Board) IRB policies from our company. The presented technology, experimental design, and results have been reviewed and further improved by feedback garnered from our internal Health Team, Product Team, User Experience (UX) Team and Legal team. Our results demonstrate that APG achieves consistently high HR (3.21% median error across 153 participants in all scenarios) and HRV (2.70% median error in interbeat interval, IBI) measurement accuracy. Our UX study further shows that APG is resilient to variation in: skin tone, sub-optimal seal conditions, and ear canal size.
View details
Preview abstract
This paper presents an evaluation of “Quick Commands” to control an Assistant in a variety of hands-free contexts. Quick Commands are an application of NLP that removes invocation words “Hey Google”, “Hey Siri”, “Alexa” can lend voice interaction to retain an element of naturalness. For situations where non-invoked commands may be useful, we anticipated media as a key opportunity for “Quick Commands” is to allow control of media through hands-free and invocation free interaction.
Research goals included validating the value proposition of Quick Commands for earbud interaction in terms of usability: feature helpfulness, naturalness, and comfort with use in public and private spaces. Study design included recruiting a group of over 80 users to test the assistant in different contexts, followed by a focus group of 18 users to provide detailed feedback. The study ran 1 week in duration and encouraged testing of Quick Commands in different scenarios: alone in a quiet room, alone with quiet, ambient noise (e.g. when walking), and alone in a noisy room (e.g. TV on.) During the week of testing, participants completed check-in surveys at the mid-week point and end of the week. A debrief session was scheduled with a random set of participants to understand commonalities and get feedback in a small group setting.
Results of this study demonstrate pain-points and delights for Quick Command and user comfort with voice interactions as part of earbud wear in private versus public settings. Evaluation methods can be replicated to validate future NLP advances and Assistant features before implementation in public facing applications.
View details
