# Cascade Residual Learning: A Two-stage Convolutional Neural Network for Stereo Matching

arXiv: https://arxiv.org/abs/1708.09204 · HTML: https://mlsanity.com/paper/1708.09204/

## Key facts

- Cited by 63 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Aug 2017.

## Built on by (most cited first, 30 of 63)

- [RAFT-Stereo: Multilevel Recurrent Field Transforms for Stereo Matching](https://mlsanity.com/paper/2109.07547/) (2021)
- [Practical Stereo Matching via Cascaded Recurrent Network with Adaptive Correlation](https://mlsanity.com/paper/2203.11483/) (2022)
- [TimeLens: Event-based Video Frame Interpolation](https://mlsanity.com/paper/2106.07286/) (2021)
- [SMD-Nets: Stereo Mixture Density Networks](https://mlsanity.com/paper/2104.03866/) (2021)
- [Stereo4D: Learning How Things Move in 3D from Internet Stereo Videos](https://mlsanity.com/paper/2412.09621/) (2024)
- [MobileStereoNet: Towards Lightweight Deep Networks for Stereo Matching](https://mlsanity.com/paper/2108.09770/) (2021)
- [LIGA-Stereo: Learning LiDAR Geometry Aware Representations for Stereo-based 3D Detector](https://mlsanity.com/paper/2108.08258/) (2021)
- [Open Challenges in Deep Stereo: the Booster Dataset](https://mlsanity.com/paper/2206.04671/) (2022)
- [Correlate-and-Excite: Real-Time Stereo Matching via Guided Cost Volume Excitation](https://mlsanity.com/paper/2108.05773/) (2021)
- [Bilateral Grid Learning for Stereo Matching Networks](https://mlsanity.com/paper/2101.01601/) (2021)
- [ITSA: An Information-Theoretic Approach to Automatic Shortcut Avoidance and Domain Generalization in Stereo Matching Networks](https://mlsanity.com/paper/2201.02263/) (2022)
- [Uncertainty Guided Adaptive Warping for Robust and Efficient Stereo Matching](https://mlsanity.com/paper/2307.14071/) (2023)
- [Learning Depth Estimation for Transparent and Mirror Surfaces](https://mlsanity.com/paper/2307.15052/) (2023)
- [Booster: a Benchmark for Depth from Images of Specular and Transparent Surfaces](https://mlsanity.com/paper/2301.08245/) (2023)
- [Multi-scale Matching Networks for Semantic Correspondence](https://mlsanity.com/paper/2108.00211/) (2021)
- [Learning Camera Localization via Dense Scene Matching](https://mlsanity.com/paper/2103.16792/) (2021)
- [Match-Stereo-Videos: Bidirectional Alignment for Consistent Dynamic Stereo Matching](https://mlsanity.com/paper/2403.10755/) (2024)
- [A Survey on Deep Stereo Matching in the Twenties](https://mlsanity.com/paper/2407.07816/) (2024)
- [Digging Into Uncertainty-based Pseudo-label for Robust Stereo Matching](https://mlsanity.com/paper/2307.16509/) (2023)
- [Neural Disparity Refinement for Arbitrary Resolution Stereo](https://mlsanity.com/paper/2110.15367/) (2021)
- [Beyond Monocular Deraining: Parallel Stereo Deraining Network Via Semantic Prior](https://mlsanity.com/paper/2105.03830/) (2021)
- [Stereoscopic Universal Perturbations across Different Architectures and Datasets](https://mlsanity.com/paper/2112.06116/) (2021)
- [TemporalStereo: Efficient Spatial-Temporal Stereo Matching Network](https://mlsanity.com/paper/2211.13755/) (2022)
- [FADNet++: Real-Time and Accurate Disparity Estimation with Configurable Networks](https://mlsanity.com/paper/2110.02582/) (2021)
- [Match Stereo Videos via Bidirectional Alignment](https://mlsanity.com/paper/2409.20283/) (2024)
- [CodedStereo: Learned Phase Masks for Large Depth-of-field Stereo](https://mlsanity.com/paper/2104.04641/) (2021)
- [ResDepth: A Deep Residual Prior For 3D Reconstruction From High-resolution Satellite Images](https://mlsanity.com/paper/2106.08107/) (2021)
- [DoubleStar: Long-Range Attack Towards Depth Estimation based Obstacle Avoidance in Autonomous Systems](https://mlsanity.com/paper/2110.03154/) (2021)
- [Active Stereo Without Pattern Projector](https://mlsanity.com/paper/2309.12315/) (2023)
- [Stereo Any Video: Temporally Consistent Stereo Matching](https://mlsanity.com/paper/2503.05549/) (2025)

## Abstract (arXiv)

Leveraging on the recent developments in convolutional neural networks (CNNs), matching dense correspondence from a stereo pair has been cast as a learning problem, with performance exceeding traditional approaches. However, it remains challenging to generate high-quality disparities for the inherently ill-posed regions. To tackle this problem, we propose a novel cascade CNN architecture composing of two stages. The first stage advances the recently proposed DispNet by equipping it with extra up-convolution modules, leading to disparity images with more details. The second stage explicitly rectifies the disparity initialized by the first stage; it couples with the first-stage and generates residual signals across multiple scales. The summation of the outputs from the two stages gives the final disparity. As opposed to directly learning the disparity at the second stage, we show that residual learning provides more effective refinement. Moreover, it also benefits the training of the overall cascade network. Experimentation shows that our cascade residual learning scheme provides state-of-the-art performance for matching stereo correspondence. By the time of the submission of this paper, our method ranks first in the KITTI 2015 stereo benchmark, surpassing the prior works by a noteworthy margin.