Google Research

Adversarially Robust Panoptic Segmentation (ARPaS) Benchmark

Adversarial Robustness in the Real World (ECCV 2022 Workshop) (to appear)


We propose the Adversarially Robust Panoptic Segmentation (ARPaS) benchmark to assess the general robustness of panoptic segmentation techniques. To account for the differences between instance and semantic segmentation, we propose to treat each segment as an independent target to optimise pixel-level adversaries. Additionally, we include common corruptions to quantify the effect of naturally occurring image perturbations in this task. We deploy the ARPaS benchmark to evaluate the robustness of state-of-the-art representatives from families of panoptic segmentation methods on standard datasets, showing their fragility in the face of attacks. To gain further insights into the effects of attacking the models, we introduce a diagnostic tool to decompose the error analysis. Finally, we empirically demonstrate that a baseline adversarial training strategy can significantly improve the robustness of these methods.

Research Areas

Learn more about how we do research

We maintain a portfolio of research projects, providing individuals and teams the freedom to emphasize specific types of work