Training Neural Networks to Produce Compatible Features
Abstract
This paper makes a first step towards compatible network components. We propose three ways which modify training to make components compatible: (i) We add a shared supervised auxiliary task which discriminates between the common classes. (ii) We add a shared self-supervised auxiliary task: rotation prediction. (iii) We initialize the networks using the same random weights.
On CIFAR-10 we show:
(i) we can train networks to produce compatible features, without degrading task accuracy compared to training the networks independently.
(ii) random initialization has a large effect on compatibility;
(ii) we can train incrementally: given previously trained components, we can train new ones which are also compatible with them
On CIFAR-10 we show:
(i) we can train networks to produce compatible features, without degrading task accuracy compared to training the networks independently.
(ii) random initialization has a large effect on compatibility;
(ii) we can train incrementally: given previously trained components, we can train new ones which are also compatible with them
Research Areas
