Google Research

Instance Embedding Transfer to Unsupervised Video Object Segmentation

Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2018)


We propose an unsupervised video object segmentation method in this work by transferring the knowledge of image-based instance embedding networks. The instance embedding networks produce an embedding for each pixel and identify all pixels belonging to the same object. It is observed that instance embeddings trained by static images are stable over consecutive video frames. Thus, we apply the trained networks to video object segmentation without model retraining or online fine-tuning and incorporate them with objectness from instance segmentation model and optical flow features. The stability of instance embedding is analyzed, and instability mitigation is studied. Our method outperforms state-of-the-art unsupervised segmentation methods in the DAVIS dataset and is competitive on the Segtrack-v2 data set.

Learn more about how we do research

We maintain a portfolio of research projects, providing individuals and teams the freedom to emphasize specific types of work