- Stanislav Fort
- Jie Jessie Ren
- Balaji Lakshminarayanan
Abstract
Near out-of-distribution detection (OOD) is a major challenge for deep neural networks. We demonstrate that large-scale pre-training can significantly improve the state-of-the-art (SOTA) on a range of near OOD tasks across different data modalities. For instance, on CIFAR-100 vs CIFAR-10 OOD detection, we improve the AUROC from 85% (current SOTA) to more than 96% using Vision Transformers pre-trained on ImageNet21k. On a challenging genomics OOD detection benchmark, we improve the AUROC from 66% (current SOTA) to 77%. To further improve performance, we explore the few-shot outlier exposure setting where a few examples from outlier classes may be available; we show that pre-trained models are well-suited to outlier exposure, and that the AUROC of OOD detection on CIFAR-100 vs CIFAR-10 can be improved to 98.7% with just 1 image per OOD class, and 99.46% with 10 images per OOD class. We observe similar trends on genomics, achieving 85% with just 1 example per OOD class. For multi-modal image-text pre-trained models such as CLIP, we explore a new way of using just the names of outlier classes as a sole source of information (without any accompanying images) and show that this outperforms previous SOTA on several standard OOD benchmark tasks.
Research Areas
Learn more about how we do research
We maintain a portfolio of research projects, providing individuals and teams the freedom to emphasize specific types of work