What makes for good views for contrastive representation learning?

Contrastive learning between multiple views of the data has recently dominated the field of self-supervised representation learning. Despite its success, the influence of different views is less studied. In this paper, we step towards understanding the importance of view selection with empirical analysis, and argue that we should reduce the mutual information (MI) between contrasted views while keeping their information bits that are relevant to the downstream task. To verify it, we devise an unsupervised and a semi-supervised framework to learn good views from the perspective of color space. We also view data augmentation as a way to reduce MI, and show that increasing data augmentation leads to decreasing MI but improved downstream classification accuracy. As a by-product, a new state-of-the-art accuracy is achieved on ImageNet linear readoff benchmark with ResNet-50.

• Machine Perception