Learning to Transfer Learn: Reinforcement Learning-Based Selection for Adaptive Transfer Learning

We propose a novel framework, named learn to transfer learn (L2TL), to improve transfer learning on a target dataset by judicious extraction of information from a source dataset. Our framework considers joint optimization of vastly-shared weights between models of source and target tasks, and employs adaptive coefficients for scaling of constituent loss terms. The adaptation of the coefficients is done using a reinforcement learning (RL)-based policy model, which is guided based on a performance metric on target evaluation set. We demonstrate the state-of-the-art performance of L2TL on various datasets, with consistent outperformance of fine-tuning baselines. Especially in the regimes of small-scale target datasets and in the cases of significant label mismatch between source and target datasets, the outperformance of L2TL is more significant.

• Machine Intelligence

• Machine Perception