Google Research

Deep Canonical Correlation Analysis for Decoding the Auditory Brain

  • Jaswanth Reddy Katthi
  • Malcolm Slaney
  • Sandeep Kothinti
  • Sriram Ganapathy
Engineering in Medicine and Biology (2020)


The process of decoding the auditory brain for an acoustic stimulus involves finding the relationship between the audio input and the brain activity measured in terms of Electroencephalography (EEG) recordings. Prior methods in this domain focus on linear analysis methods like Canonical Correlation Analysis (CCA) to establish this relationship. In this paper, we present a deep learning framework which is learned to maximize a correlation cost. For dealing with high levels of noise in EEG data, we employ regularization techniques and experiment with various model architectures. With a paired dataset of audio envelope and EEG, we perform several experiments with deep correlation analysis using forward and backward correlation models. In these experiments, we show that the deep CCA is consistently able to outperform the linear models in terms of providing improved correlation (up to 9% absolute improvement in Pearson correlation which is statistically significant). We also present an analysis that highlights the benefits of using dropouts for neural network regularization in the deep CCA model.

Research Areas

Learn more about how we do research

We maintain a portfolio of research projects, providing individuals and teams the freedom to emphasize specific types of work