Google Research

Parametric Spectral Filters for Fast Converging,Scalable Convolutional Neural Networks

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, pp. 2800-2804 (to appear)

Abstract

Abstract—Using spectral multiplication to compute convolution in neural networks has been investigated by a number of researchers because of its potential in speeding up computations for large images. However, previous methods require the learning of arbitrarily large convolution filters in the spectral domain, causing two untenable problems: an explosion in the number of trainable parameters per filter and an inability to reuse filters across images of differing sizes. To address this, we propose the usage of spectral approximation functions to approximate the massive Spectral domain filters with only a few trainable parameters. Our empirical analysis suggests that the proposed approximation maintains the benefits of arbitrarily large filters(such as improved rate of convergence in training, accuracy, and stability) while relying on significantly fewer trainable parameters.

Learn more about how we do research

We maintain a portfolio of research projects, providing individuals and teams the freedom to emphasize specific types of work