# Adaptive Multi-Modal Cross-Entropy Loss for Stereo Matching

arXiv: https://arxiv.org/abs/2306.15612 · HTML: https://mlsanity.com/paper/2306.15612/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 40 references that are in the map.
- First posted on arXiv 27 Jun 2023.

## Builds on (most cited first, 28 of 28)

- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation](https://mlsanity.com/paper/1512.02134/) (2015)
- [PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization](https://mlsanity.com/paper/1905.05172/) (2019)
- [Deep Learning and the Information Bottleneck Principle](https://mlsanity.com/paper/1503.02406/) (2015)
- [Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition](https://mlsanity.com/paper/1406.4729/) (2014)
- [Pyramid Stereo Matching Network](https://mlsanity.com/paper/1803.08669/) (2018)
- [End-to-End Learning of Geometry and Context for Deep Stereo Regression](https://mlsanity.com/paper/1703.04309/) (2017)
- [RAFT-Stereo: Multilevel Recurrent Field Transforms for Stereo Matching](https://mlsanity.com/paper/2109.07547/) (2021)
- [GA-Net: Guided Aggregation Net for End-to-end Stereo Matching](https://mlsanity.com/paper/1904.06587/) (2019)
- [Group-wise Correlation Stereo Network](https://mlsanity.com/paper/1903.04025/) (2019)
- [Practical Stereo Matching via Cascaded Recurrent Network with Adaptive Correlation](https://mlsanity.com/paper/2203.11483/) (2022)
- [Iterative Geometry Encoding Volume for Stereo Matching](https://mlsanity.com/paper/2303.06615/) (2023)
- [Hierarchical Neural Architecture Search for Deep Stereo Matching](https://mlsanity.com/paper/2010.13501/) (2020)
- [CroCo v2: Improved Cross-view Completion Pre-training for Stereo Matching and Optical Flow](https://mlsanity.com/paper/2211.10408/) (2022)
- [CFNet: Cascade and Fused Cost Volume for Robust Stereo Matching](https://mlsanity.com/paper/2104.04314/) (2021)
- [Attention Concatenation Volume for Accurate and Efficient Stereo Matching](https://mlsanity.com/paper/2203.02146/) (2022)
- [SMD-Nets: Stereo Mixture Density Networks](https://mlsanity.com/paper/2104.03866/) (2021)
- [Domain-invariant Stereo Matching Networks](https://mlsanity.com/paper/1911.13287/) (2019)
- [End-to-End Pseudo-LiDAR for Image-Based 3D Object Detection](https://mlsanity.com/paper/2004.03080/) (2020)
- [Adaptive Unimodal Cost Volume Filtering for Deep Stereo Matching](https://mlsanity.com/paper/1909.03751/) (2019)
- [Revisiting Domain Generalized Stereo Matching Networks from a Feature Consistency Perspective](https://mlsanity.com/paper/2203.10887/) (2022)
- [GraftNet: Towards Domain Generalized Stereo Matching with a Broad-Spectrum and Task-Oriented Feature](https://mlsanity.com/paper/2204.00179/) (2022)
- [ITSA: An Information-Theoretic Approach to Automatic Shortcut Avoidance and Domain Generalization in Stereo Matching Networks](https://mlsanity.com/paper/2201.02263/) (2022)
- [Wasserstein Distances for Stereo Disparity Estimation](https://mlsanity.com/paper/2007.03085/) (2020)
- [Local Similarity Pattern and Cost Self-Reassembling for Deep Stereo Matching Networks](https://mlsanity.com/paper/2112.01011/) (2021)
- [Practical Deep Stereo (PDS): Toward applications-friendly deep stereo matching](https://mlsanity.com/paper/1806.01677/) (2018)
- [Non-parametric Depth Distribution Modelling based Depth Inference for Multi-view Stereo](https://mlsanity.com/paper/2205.03783/) (2022)

## Built on by (most cited first, 19 of 19)

- [IGEV++: Iterative Multi-range Geometry Encoding Volumes for Stereo Matching](https://mlsanity.com/paper/2409.00638/) (2024)
- [DEFOM-Stereo: Depth Foundation Model Based Stereo Matching](https://mlsanity.com/paper/2501.09466/) (2025)
- [Stereo Anywhere: Robust Zero-Shot Deep Stereo Matching Even Where Either Stereo or Mono Fail](https://mlsanity.com/paper/2412.04472/) (2024)
- [OpenStereo: A Comprehensive Benchmark for Stereo Matching and Strong Baseline](https://mlsanity.com/paper/2312.00343/) (2023)
- [LightStereo: Channel Boost Is All You Need for Efficient 2D Cost Aggregation](https://mlsanity.com/paper/2406.19833/) (2024)
- [A Survey on Deep Stereo Matching in the Twenties](https://mlsanity.com/paper/2407.07816/) (2024)
- [Motif Channel Opened in a White-Box: Stereo Matching via Motif Correlation Graph](https://mlsanity.com/paper/2411.12426/) (2024)
- [Hadamard Attention Recurrent Transformer: A Strong Baseline for Stereo Matching Transformer](https://mlsanity.com/paper/2501.01023/) (2025)
- [PanMatch: Unleashing the Potential of Large Vision Models for Unified Matching Models](https://mlsanity.com/paper/2507.08400/) (2025)
- [SR-Stereo & DAPE: Stepwise Regression and Pre-trained Edges for Practical Stereo Matching](https://mlsanity.com/paper/2406.06953/) (2024)
- [MaDis-Stereo: Enhanced Stereo Matching via Distilled Masked Image Modeling](https://mlsanity.com/paper/2409.02846/) (2024)
- [The Sampling-Gaussian for stereo matching](https://mlsanity.com/paper/2410.06527/) (2024)
- [MIDAS: Modeling Ground-Truth Distributions with Dark Knowledge for Domain Generalized Stereo Matching](https://mlsanity.com/paper/2503.04376/) (2025)
- [CMD: Constraining Multimodal Distribution for Domain Adaptation in Stereo Matching](https://mlsanity.com/paper/2504.21302/) (2025)
- [MAGIC: Learning from Visibility Asymmetry for Unsupervised Stereo Matching](https://mlsanity.com/paper/2508.10838/) (2025)
- [MatchAttention: Embedding Explicit Matching Constraints into Attention for Efficient Stereo Matching](https://mlsanity.com/paper/2510.14260/) (2025)
- [StereoFactory: A Unified Merging Framework for Robust Stereo Matching](https://mlsanity.com/paper/2606.17475/) (2026)
- [LiteMatch: Lightweight Zero-Shot Stereo Matching via Cost Volume Stabilization](https://mlsanity.com/paper/2606.31636/) (2026)
- [STEREOFLOW: Progressive Stereo Matching with StereoDiT and Transition Flow Matching](https://mlsanity.com/paper/2607.19986/) (2026)

## Abstract (arXiv)

Despite the great success of deep learning in stereo matching, recovering accurate disparity maps is still challenging. Currently, L1 and cross-entropy are the two most widely used losses for stereo network training. Compared with the former, the latter usually performs better thanks to its probability modeling and direct supervision to the cost volume. However, how to accurately model the stereo ground-truth for cross-entropy loss remains largely under-explored. Existing works simply assume that the ground-truth distributions are uni-modal, which ignores the fact that most of the edge pixels can be multi-modal. In this paper, a novel adaptive multi-modal cross-entropy loss (ADL) is proposed to guide the networks to learn different distribution patterns for each pixel. Moreover, we optimize the disparity estimator to further alleviate the bleeding or misalignment artifacts in inference. Extensive experimental results show that our method is generic and can help classic stereo networks regain state-of-the-art performance. In particular, GANet with our method ranks 1st on both the KITTI 2015 and 2012 benchmarks among the published methods. Meanwhile, excellent synthetic-to-realistic generalization performance can be achieved by simply replacing the traditional loss with ours.