# Level Set Binocular Stereo with Occlusions

arXiv: https://arxiv.org/abs/2109.03464 · HTML: https://mlsanity.com/paper/2109.03464/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 47 references that are in the map.
- First posted on arXiv 8 Sep 2021.

## Builds on (most cited first, 8 of 8)

- [Holistically-Nested Edge Detection](https://mlsanity.com/paper/1504.06375/) (2015)
- [Stereo Matching by Training a Convolutional Neural Network to Compare Image Patches](https://mlsanity.com/paper/1510.05970/) (2015)
- [Hierarchical Deep Stereo Matching on High-resolution Images](https://mlsanity.com/paper/1912.06704/) (2019)
- [Falling Things: A Synthetic Dataset for 3D Object Detection and Pose Estimation](https://mlsanity.com/paper/1804.06534/) (2018)
- [EdgeStereo: An Effective Multi-Task Learning Network for Stereo Matching and Edge Detection](https://mlsanity.com/paper/1903.01700/) (2019)
- [On the Synergies between Machine Learning and Binocular Stereo for Depth Estimation from Images: a Survey](https://mlsanity.com/paper/2004.08566/) (2020)
- [Low-level Vision by Consensus in a Spatial Hierarchy of Regions](https://mlsanity.com/paper/1411.4894/) (2014)
- [Level Set Stereo for Cooperative Grouping with Occlusion](https://mlsanity.com/paper/2006.16094/) (2020)

## Abstract (arXiv)

Localizing stereo boundaries and predicting nearby disparities are difficult because stereo boundaries induce occluded regions where matching cues are absent. Most modern computer vision algorithms treat occlusions secondarily (e.g., via left-right consistency checks after matching) or rely on high-level cues to improve nearby disparities (e.g., via deep networks and large training sets). They ignore the geometry of stereo occlusions, which dictates that the spatial extent of occlusion must equal the amplitude of the disparity jump that causes it. This paper introduces an energy and level-set optimizer that improves boundaries by encoding occlusion geometry. Our model applies to two-layer, figure-ground scenes, and it can be implemented cooperatively using messages that pass predominantly between parents and children in an undecimated hierarchy of multi-scale image patches. In a small collection of figure-ground scenes curated from Middlebury and Falling Things stereo datasets, our model provides more accurate boundaries than previous occlusion-handling stereo techniques. This suggests new directions for creating cooperative stereo systems that incorporate occlusion cues in a human-like manner.