# Fine-tuning deep learning models for stereo matching using results from semi-global matching

arXiv: https://arxiv.org/abs/2205.14051 · HTML: https://mlsanity.com/paper/2205.14051/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 22 references that are in the map.
- First posted on arXiv 27 May 2022.

## Builds on (most cited first, 11 of 11)

- [A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation](https://mlsanity.com/paper/1512.02134/) (2015)
- [Pyramid Stereo Matching Network](https://mlsanity.com/paper/1803.08669/) (2018)
- [End-to-End Learning of Geometry and Context for Deep Stereo Regression](https://mlsanity.com/paper/1703.04309/) (2017)
- [Hierarchical Neural Architecture Search for Deep Stereo Matching](https://mlsanity.com/paper/2010.13501/) (2020)
- [Computing the Stereo Matching Cost with a Convolutional Neural Network](https://mlsanity.com/paper/1409.4326/) (2014)
- [Self-Supervised Learning for Stereo Matching with Self-Improving Ability](https://mlsanity.com/paper/1709.00930/) (2017)
- [AdaStereo: A Simple and Efficient Approach for Adaptive Stereo Matching](https://mlsanity.com/paper/2004.04627/) (2020)
- [Zoom and Learn: Generalizing Deep Stereo Matching to Novel Domains](https://mlsanity.com/paper/1803.06641/) (2018)
- [Stereo Matching by Self-supervision of Multiscopic Vision](https://mlsanity.com/paper/2104.04170/) (2021)
- [A Survey on Deep Learning Architectures for Image-based Depth Reconstruction](https://mlsanity.com/paper/1906.06113/) (2019)
- [Self-Supervised Learning for Stereo Reconstruction on Aerial Images](https://mlsanity.com/paper/1907.12446/) (2019)

## Abstract (arXiv)

Deep learning (DL) methods are widely investigated for stereo image matching tasks due to their reported high accuracies. However, their transferability/generalization capabilities are limited by the instances seen in the training data. With satellite images covering large-scale areas with variances in locations, content, land covers, and spatial patterns, we expect their performances to be impacted. Increasing the number and diversity of training data is always an option, but with the ground-truth disparity being limited in remote sensing due to its high cost, it is almost impossible to obtain the ground-truth for all locations. Knowing that classical stereo matching methods such as Census-based semi-global-matching (SGM) are widely adopted to process different types of stereo data, we therefore, propose a finetuning method that takes advantage of disparity maps derived from SGM on target stereo data. Our proposed method adopts a simple scheme that uses the energy map derived from the SGM algorithm to select high confidence disparity measurements, at the same utilizing the images to limit these selected disparity measurements on texture-rich regions. Our approach aims to investigate the possibility of improving the transferability of current DL methods to unseen target data without having their ground truth as a requirement. To perform a comprehensive study, we select 20 study-sites around the world to cover a variety of complexities and densities. We choose well-established DL methods like geometric and context network (GCNet), pyramid stereo matching network (PSMNet), and LEAStereo for evaluation. Our results indicate an improvement in the transferability of the DL methods across different regions visually and numerically.