# PVStereo: Pyramid Voting Module for End-to-End Self-Supervised Stereo Matching

arXiv: https://arxiv.org/abs/2103.07094 · HTML: https://mlsanity.com/paper/2103.07094/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 42 references that are in the map.
- First posted on arXiv 12 Mar 2021.

## Builds on (most cited first, 26 of 26)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [Unsupervised Monocular Depth Estimation with Left-Right Consistency](https://mlsanity.com/paper/1609.03677/) (2016)
- [Pyramid Stereo Matching Network](https://mlsanity.com/paper/1803.08669/) (2018)
- [Group-wise Correlation Stereo Network](https://mlsanity.com/paper/1903.04025/) (2019)
- [Stereo Matching by Training a Convolutional Neural Network to Compare Image Patches](https://mlsanity.com/paper/1510.05970/) (2015)
- [Learning by Analogy: Reliable Supervision from Transformations for Unsupervised Optical Flow Estimation](https://mlsanity.com/paper/2003.13045/) (2020)
- [HITNet: Hierarchical Iterative Tile Refinement Network for Real-time Stereo Matching](https://mlsanity.com/paper/2007.12140/) (2020)
- [SNE-RoadSeg: Incorporating Surface Normal Information into Semantic Segmentation for Accurate Freespace Detection](https://mlsanity.com/paper/2008.11351/) (2020)
- [Adaptive Unimodal Cost Volume Filtering for Deep Stereo Matching](https://mlsanity.com/paper/1909.03751/) (2019)
- [Anytime Stereo Image Depth Estimation on Mobile Devices](https://mlsanity.com/paper/1810.11408/) (2018)
- [Flow2Stereo: Effective Self-Supervised Learning of Optical Flow and Stereo Matching](https://mlsanity.com/paper/2004.02138/) (2020)
- [Road surface 3d reconstruction based on dense subpixel disparity map estimation](https://mlsanity.com/paper/1807.01874/) (2018)
- [Self-Supervised Learning for Stereo Matching with Self-Improving Ability](https://mlsanity.com/paper/1709.00930/) (2017)
- [Rethinking Road Surface 3D Reconstruction and Pothole Detection: From Perspective Transformation to Disparity Map Segmentation](https://mlsanity.com/paper/2012.10802/) (2020)
- [Dynamic Fusion Module Evolves Drivable Area and Road Anomaly Detection: A Benchmark and Algorithms](https://mlsanity.com/paper/2103.02433/) (2021)
- [Applying Surface Normal Information in Drivable Area and Road Anomaly Detection for Ground Mobile Robots](https://mlsanity.com/paper/2008.11383/) (2020)
- [Learning Collision-Free Space Detection from Stereo Images: Homography Matrix Brings Better Data Augmentation](https://mlsanity.com/paper/2012.07890/) (2020)
- [On the Importance of Stereo for Accurate Depth Estimation: An Efficient Semi-Supervised Deep Neural Network Approach](https://mlsanity.com/paper/1803.09719/) (2018)
- [Self-Supervised Drivable Area and Road Anomaly Segmentation using RGB-D Data for Robotic Wheelchairs](https://mlsanity.com/paper/2007.05950/) (2020)
- [Computer Stereo Vision for Autonomous Driving](https://mlsanity.com/paper/2012.03194/) (2020)
- [Fast Deep Stereo with 2D Convolutional Processing of Cost Signatures](https://mlsanity.com/paper/1903.04939/) (2019)
- [We Learn Better Road Pothole Detection: from Attention Aggregation to Adversarial Domain Adaptation](https://mlsanity.com/paper/2008.06840/) (2020)

## Built on by (most cited first, 28 of 28)

- [RAFT-Stereo: Multilevel Recurrent Field Transforms for Stereo Matching](https://mlsanity.com/paper/2109.07547/) (2021)
- [Iterative Geometry Encoding Volume for Stereo Matching](https://mlsanity.com/paper/2303.06615/) (2023)
- [OpenStereo: A Comprehensive Benchmark for Stereo Matching and Strong Baseline](https://mlsanity.com/paper/2312.00343/) (2023)
- [LightStereo: Channel Boost Is All You Need for Efficient 2D Cost Aggregation](https://mlsanity.com/paper/2406.19833/) (2024)
- [Learning Interpretable End-to-End Vision-Based Motion Planning for Autonomous Driving with Optical Flow Distillation](https://mlsanity.com/paper/2104.12861/) (2021)
- [SNE-RoadSeg+: Rethinking Depth-Normal Translation and Deep Supervision for Freespace Detection](https://mlsanity.com/paper/2107.14599/) (2021)
- [Guided Depth Map Super-resolution: A Survey](https://mlsanity.com/paper/2302.09598/) (2023)
- [Computer Vision for Road Imaging and Pothole Detection: A State-of-the-Art Review of Systems and Algorithms](https://mlsanity.com/paper/2204.13590/) (2022)
- [Robust Synthetic-to-Real Transfer for Stereo Matching](https://mlsanity.com/paper/2403.07705/) (2024)
- [ChiTransformer:Towards Reliable Stereo from Cues](https://mlsanity.com/paper/2203.04554/) (2022)
- [Learning Intra-view and Cross-view Geometric Knowledge for Stereo Matching](https://mlsanity.com/paper/2402.19270/) (2024)
- [An Improved RaftStereo Trained with A Mixed Dataset for the Robust Vision Challenge 2022](https://mlsanity.com/paper/2210.12785/) (2022)
- [These Maps Are Made by Propagation: Adapting Deep Stereo Networks to Road Scenarios with Decisive Disparity Diffusion](https://mlsanity.com/paper/2411.03717/) (2024)
- [SCV-Stereo: Learning Stereo Matching from a Sparse Cost Volume](https://mlsanity.com/paper/2107.08187/) (2021)
- [Dive Deeper into Rectifying Homography for Stereo Camera Online Self-Calibration](https://mlsanity.com/paper/2309.10314/) (2023)
- [Rethinking Iterative Stereo Matching from Diffusion Bridge Model Perspective](https://mlsanity.com/paper/2404.09051/) (2024)
- [Computer-Aided Road Inspection: Systems and Algorithms](https://mlsanity.com/paper/2203.02355/) (2022)
- [Co-Teaching: An Ark to Unsupervised Stereo Matching](https://mlsanity.com/paper/2107.08186/) (2021)
- [Sparse LiDAR Assisted Self-supervised Stereo Disparity Estimation](https://mlsanity.com/paper/2112.15355/) (2021)
- [Non-learning Stereo-aided Depth Completion under Mis-projection via Selective Stereo Matching](https://mlsanity.com/paper/2210.01436/) (2022)
- [The Sampling-Gaussian for stereo matching](https://mlsanity.com/paper/2410.06527/) (2024)
- [Rectified Iterative Disparity for Stereo Matching](https://mlsanity.com/paper/2406.10943/) (2024)
- [SAda-Net: A Self-Supervised Adaptive Stereo Estimation CNN For Remote Sensing Image Data](https://mlsanity.com/paper/2410.13500/) (2024)
- [RGB-Phase Speckle: Cross-Scene Stereo 3D Reconstruction via Wrapped Pre-Normalization](https://mlsanity.com/paper/2503.06125/) (2025)
- [A Reinforcement Learning Method for Environments with Stochastic Variables: Post-Decision Proximal Policy Optimization with Dual Critic Networks](https://mlsanity.com/paper/2504.05150/) (2025)
- [Integrating Disparity Confidence Estimation into Relative Depth Prior-Guided Unsupervised Stereo Matching](https://mlsanity.com/paper/2508.01275/) (2025)
- [CCNeXt: An Effective Self-Supervised Stereo Depth Estimation Approach](https://mlsanity.com/paper/2509.22627/) (2025)
- [Geometry Reinforced Efficient Attention Tuning Equipped with Normals for Robust Stereo Matching](https://mlsanity.com/paper/2604.09142/) (2026)

## Abstract (arXiv)

Supervised learning with deep convolutional neural networks (DCNNs) has seen huge adoption in stereo matching. However, the acquisition of large-scale datasets with well-labeled ground truth is cumbersome and labor-intensive, making supervised learning-based approaches often hard to implement in practice. To overcome this drawback, we propose a robust and effective self-supervised stereo matching approach, consisting of a pyramid voting module (PVM) and a novel DCNN architecture, referred to as OptStereo. Specifically, our OptStereo first builds multi-scale cost volumes, and then adopts a recurrent unit to iteratively update disparity estimations at high resolution; while our PVM can generate reliable semi-dense disparity images, which can be employed to supervise OptStereo training. Furthermore, we publish the HKUST-Drive dataset, a large-scale synthetic stereo dataset, collected under different illumination and weather conditions for research purposes. Extensive experimental results demonstrate the effectiveness and efficiency of our self-supervised stereo matching approach on the KITTI Stereo benchmarks and our HKUST-Drive dataset. PVStereo, our best-performing implementation, greatly outperforms all other state-of-the-art self-supervised stereo matching approaches. Our project page is available at sites.google.com/view/pvstereo.