A dataset and model for auditory scene recognition for hearing devices: AHEAD-DS and OpenYAMNet

Scene recognition is important for hearing devices; however, this is challenging, in part
because of the limitations of existing datasets. Datasets often lack public accessibility,
non-reproducible composition, or audiologically relevant labels, hindering systematic
comparison of machine learning models. Deploying such models on
resource-constrained edge devices presents another challenge. The proposed solution
is two-fold, a repackaging and refinement of several open source datasets to create
AHEAD-DS, a dataset designed for auditory scene recognition for hearing devices, and
introduce OpenYAMNet, a fully open source re-implementation of YAMNet. AHEAD-DS
aims to provide a standardised, publicly available dataset with labels relevant to hearing
aids, facilitating model comparison. OpenYAMNet is designed for deployment on edge
devices like smartphones connected to hearing devices, such as hearing aids and
wireless earphones with hearing aid functionality, serving as a baseline model for
sound-based scene recognition. OpenYAMNet achieved a mean average precision of
0.86 and accuracy of 0.93 on the testing set of AHEAD-DS across fourteen categories
relevant to auditory scene recognition. Tests on TAU 2019 (unseen data) show good
alignment between predicted and expected results. Real-time sound-based scene
recognition capabilities were demonstrated on edge devices by deploying OpenYAMNet
to an Android smartphone. Even with a 2018 Google Pixel 3, a phone with modest
specifications, the model processes audio with approximately 50ms of latency to load
the model, and an approximate linear increase of 30ms per 1s of audio. The project
website with links to code, data, and models.
\href{https://github.com/Australian-Future-Hearing-Initiative}{https://github.com/Australia
n-Future-Hearing-Initiative}