Neural network feature visualization is a powerful technique. It can answer questions about what a network — or parts of a network — are looking for by generating idealized examples of what the network is trying to ﬁnd.
Over the last few years, the ﬁeld has made great strides in feature visualization. Actually getting it to work, however, involves a number of details. In this article, we examine the major issues and explore common approaches to solving them.
We ﬁnd that remarkably simple methods can produce state-of-the-art visualizations — and that, surprisingly, these visualizations are often limited by optimization problems that can be solved with standard techniques.