Google Research

ESPReSSo: Efficient Slanted PatchMatch for Real-time Spacetime Stereo

In Proceedings of Sixth International Conference on 3D Vision (3DV) (2018) (to appear)

Abstract

We present ESPReSSo, the first real-time implementation of spacetime stereo, offering improved quality vs. existing real-time systems. ESPReSSo uses a local stereo reconstruction algorithm that precomputes subpixel-shifted binary descriptors, then iteratively samples those descriptors along slanted disparity plane hypotheses, applying an edge-aware filter for spatial cost aggregation. Plane hypotheses are shared across rectangular tiles, but every pixel gets a different winner, much as in PatchMatch Filter. This architecture performs very few descriptor computations but many cost aggregations, and we tune our choice of descriptor and filter accordingly: We propose a new 32-bit binary spacetime descriptor breve that combines the benefits of small spatial extent with robustness to scene motion, and the system aggregates costs using the permeability filter, a very efficient edge-aware filter. Our prototype system outputs 60 depth frames per second on a desktop GPU, using less than 11ms total computation per frame.

Research Areas

Learn more about how we do research

We maintain a portfolio of research projects, providing individuals and teams the freedom to emphasize specific types of work