Learning Fast Sample Re-weighting Without Reward Data
Abstract
Re-weighting training samples has been shown as an effective and practical approach to tackle data biases such as imbalance and corrupted labels. Recent methods develop learning based algorithms
to learn re-weighting strategies jointly with model training, in light of reinforcement learning and meta learning. However, the dependence of additional unbiased reward data is known an undesirable limitation. Furthermore, existing learning based sample weighting methods maintain inner and outer optimization for model and weighting parameters, respectively, such that training requires expensive optimization. This paper aims to address these two problems and presents a new learning based fast sample re-weighting (FSR) method without reward data. The method is based on two key ideas, a) learning from history as dictionary fetch and b) feature sharing. Without the dependence of constructing extra reward datasets, we can easily incorporate FSR with additionally proposed task-specific components and test on label noise robust and long-tailed recognition benchmarks. Our experiments show the proposed method achieves competitive results to state-of-the-art methods in respective tasks and significantly improved training efficiency. Source code will be released.
to learn re-weighting strategies jointly with model training, in light of reinforcement learning and meta learning. However, the dependence of additional unbiased reward data is known an undesirable limitation. Furthermore, existing learning based sample weighting methods maintain inner and outer optimization for model and weighting parameters, respectively, such that training requires expensive optimization. This paper aims to address these two problems and presents a new learning based fast sample re-weighting (FSR) method without reward data. The method is based on two key ideas, a) learning from history as dictionary fetch and b) feature sharing. Without the dependence of constructing extra reward datasets, we can easily incorporate FSR with additionally proposed task-specific components and test on label noise robust and long-tailed recognition benchmarks. Our experiments show the proposed method achieves competitive results to state-of-the-art methods in respective tasks and significantly improved training efficiency. Source code will be released.
Research Areas
