Improving Open Set Domain Adaptation Using Image-to-Image Translation

Domain adaptation has become a promising way to address the data scarcity issue in visual recognition, by adapting a model from a label-rich (source) domain to a different but related label-scarce (target) domain. While most approaches consider only an ideal \textit{closed set} scenario where both domains contain exactly the same set of known classes, we try to address the more realistic and challenging \textit{open set} domain adaptation problem where the target domain has samples belonging to classes not existing in the source, which ends up being an additional class, collectively called the unknown class. The open set problem was rarely studied and its existing solutions are mostly based on learning a joint latent space which may encounter issues when the domains differ significantly from each other. This work is driven by the question whether or not it is beneficial to operate the source images to another image domain as close to the target as possible. We propose to address the open set domain adaptation problem by aligning sample at both feature space and pixel space. Our approach, called Open Set Translation and Adaptation Network (\textsc{Ostan}), consists of two main components: translation and adaptation. The translation model is a cycle-consistent generative adversarial network, which translates any source sample to the "style" of a target domain. The adaptation network is built upon OpenBP, an open set domain adaptation framework, and trained using both (labeled) translated source images and (unlabeled) target images. The proposed \textsc{Ostan} model significantly outperforms the state-of-the-art open set domain adaptation methods on multiple public datasets. Our experiment also demonstrates that an image-to-image translation component can further improve the decision boundaries for both known and unknown classes.

• Machine Perception

• Machine Intelligence