The Differentiable Auditory Loop (DAL): An ML Framework for Hyper-Personalized Hearing Aids Using Biological Sound Representations
Abstract
Conventional hearing aids rely on fixed, frequency-dependent amplification and
compression to manage reduced sensitivity, which often fails to provide suffi-
cient listening support in complex environments, such as situations with multiple
speakers (the “cocktail party” problem). To more comprehensively address the
underlying encoding dysfunctions of hearing loss, we introduce the Differentiable
Auditory Loop (DAL), a new open-source framework for personalized hearing aid
design and fitting. Our first implementation of DAL incorporates CARFAC, a dif-
ferentiable model of human cochlear function, which we ported to JAX, to optimize
a deep neural network to match impaired auditory neural activity patterns with a
normal-hearing reference. To build a hearing aid with the fine-grained spectro-
temporal signal processing required, we adopt SEANet, a waveform-to-waveform
fully convolutional UNet generator. We fine-tune the network by comparing the
outputs of a CARFAC model fitted to normal hearing with that of a CARFAC model
fitted to match each subject’s individual hearing impairment. The comparison is
done using loss functions derived from the respective CARFAC neural activity
pattern (NAP) outputs and stabilized auditory images (SAIs), the latter providing
a 2D representation that captures phase-insensitive temporal structure in the au-
ditory nerve output. Through gradient descent, the SEANet model learns to both
denoise the input and compensate for the hearing loss modelled by the impaired
CARFAC model. Across neural-representation and signal-fidelity metrics, the
DAL-optimized SEANet model outperformed the tested master hearing aid (MHA)
baselines. The DAL framework provides a practical path toward model-based,
machine-learning-driven personalization of hearing aid signal processing. Next
steps include hardware deployment to enable real-world clinical testing.
compression to manage reduced sensitivity, which often fails to provide suffi-
cient listening support in complex environments, such as situations with multiple
speakers (the “cocktail party” problem). To more comprehensively address the
underlying encoding dysfunctions of hearing loss, we introduce the Differentiable
Auditory Loop (DAL), a new open-source framework for personalized hearing aid
design and fitting. Our first implementation of DAL incorporates CARFAC, a dif-
ferentiable model of human cochlear function, which we ported to JAX, to optimize
a deep neural network to match impaired auditory neural activity patterns with a
normal-hearing reference. To build a hearing aid with the fine-grained spectro-
temporal signal processing required, we adopt SEANet, a waveform-to-waveform
fully convolutional UNet generator. We fine-tune the network by comparing the
outputs of a CARFAC model fitted to normal hearing with that of a CARFAC model
fitted to match each subject’s individual hearing impairment. The comparison is
done using loss functions derived from the respective CARFAC neural activity
pattern (NAP) outputs and stabilized auditory images (SAIs), the latter providing
a 2D representation that captures phase-insensitive temporal structure in the au-
ditory nerve output. Through gradient descent, the SEANet model learns to both
denoise the input and compensate for the hearing loss modelled by the impaired
CARFAC model. Across neural-representation and signal-fidelity metrics, the
DAL-optimized SEANet model outperformed the tested master hearing aid (MHA)
baselines. The DAL framework provides a practical path toward model-based,
machine-learning-driven personalization of hearing aid signal processing. Next
steps include hardware deployment to enable real-world clinical testing.
