Non-discriminative data or weak model? On the relative importance of data and model resolution
Abstract
We explore the question of how the resolution of input image affects the performance of a neural network when compared to the resolution of hidden layers. Image resolution is frequently used as a hyper parameter providing a trade-off between model performance and accuracy. An intuitive interpretation is that the decay in accuracy when reducing input resolution, is caused by the reduced information content in the low-resolution input. Left unsaid often the fact that this also reduces the model's internal resolution. In this paper, we show
that up-to a point the resolution plays very little role in the network performance. We show that another obvious hypothesis, such as changes in receptive fields, is not the primary root causes either. We then use this insight, to develop novel neural network architectures that we call {\it isometric neural networks} that maintain fixed internal resolution throughout their entire depth and demonstrate that it lead of high accuracy models with low activation footprint and a parameter count.
\end{abstract}
that up-to a point the resolution plays very little role in the network performance. We show that another obvious hypothesis, such as changes in receptive fields, is not the primary root causes either. We then use this insight, to develop novel neural network architectures that we call {\it isometric neural networks} that maintain fixed internal resolution throughout their entire depth and demonstrate that it lead of high accuracy models with low activation footprint and a parameter count.
\end{abstract}
Research Areas
