# H-Net: Unsupervised Attention-based Stereo Depth Estimation Leveraging Epipolar Geometry

arXiv: https://arxiv.org/abs/2104.11288 · HTML: https://mlsanity.com/paper/2104.11288/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 48 references that are in the map.
- First posted on arXiv 22 Apr 2021.

## Builds on (most cited first, 30 of 36)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [Spatial Transformer Networks](https://mlsanity.com/paper/1506.02025/) (2015)
- [Depth Map Prediction from a Single Image using a Multi-Scale Deep Network](https://mlsanity.com/paper/1406.2283/) (2014)
- [Unsupervised Monocular Depth Estimation with Left-Right Consistency](https://mlsanity.com/paper/1609.03677/) (2016)
- [Unsupervised Learning of Depth and Ego-Motion from Video](https://mlsanity.com/paper/1704.07813/) (2017)
- [Deep Ordinal Regression Network for Monocular Depth Estimation](https://mlsanity.com/paper/1806.02446/) (2018)
- [Pyramid Stereo Matching Network](https://mlsanity.com/paper/1803.08669/) (2018)
- [Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescue](https://mlsanity.com/paper/1603.04992/) (2016)
- [End-to-End Learning of Geometry and Context for Deep Stereo Regression](https://mlsanity.com/paper/1703.04309/) (2017)
- [Virtual Worlds as Proxy for Multi-Object Tracking Analysis](https://mlsanity.com/paper/1605.06457/) (2016)
- [D3VO: Deep Depth, Deep Pose and Deep Uncertainty for Monocular Visual Odometry](https://mlsanity.com/paper/2003.01060/) (2020)
- [AANet: Adaptive Aggregation Network for Efficient Stereo Matching](https://mlsanity.com/paper/2004.09548/) (2020)
- [Learning Depth from Single Monocular Images Using Deep Convolutional Neural Fields](https://mlsanity.com/paper/1502.07411/) (2015)
- [Unsupervised Learning of Monocular Depth Estimation and Visual Odometry with Deep Feature Reconstruction](https://mlsanity.com/paper/1803.03893/) (2018)
- [Learning Depth from Monocular Videos using Direct Methods](https://mlsanity.com/paper/1712.00175/) (2017)
- [Self-supervised Monocular Trained Depth Estimation using Self-attention and Discrete Disparity Volume](https://mlsanity.com/paper/2003.13951/) (2020)
- [DeepPruner: Learning Efficient Stereo Matching via Differentiable PatchMatch](https://mlsanity.com/paper/1909.05845/) (2019)
- [Semi-Supervised Deep Learning for Monocular Depth Map Prediction](https://mlsanity.com/paper/1702.02706/) (2017)
- [Every Pixel Counts ++: Joint Learning of Geometry and Motion with 3D Holistic Understanding](https://mlsanity.com/paper/1810.06125/) (2018)
- [UnDeepVO: Monocular Visual Odometry through Unsupervised Deep Learning](https://mlsanity.com/paper/1709.06841/) (2017)
- [Deep Virtual Stereo Odometry: Leveraging Deep Depth Prediction for Monocular Direct Sparse Odometry](https://mlsanity.com/paper/1807.02570/) (2018)
- [Learning Monocular Depth by Distilling Cross-domain Stereo Networks](https://mlsanity.com/paper/1808.06586/) (2018)
- [SuperDepth: Self-Supervised, Super-Resolved Monocular Depth Estimation](https://mlsanity.com/paper/1810.01849/) (2018)
- [AdaDepth: Unsupervised Content Congruent Adaptation for Depth Estimation](https://mlsanity.com/paper/1803.01599/) (2018)
- [Normal Assisted Stereo Depth Estimation](https://mlsanity.com/paper/1911.10444/) (2019)

## Built on by (most cited first, 12 of 12)

- [Self-Supervised Generative Adversarial Network for Depth Estimation in Laparoscopic Images](https://mlsanity.com/paper/2107.04644/) (2021)
- [Self-Supervised Depth Estimation in Laparoscopic Image using 3D Geometric Consistency](https://mlsanity.com/paper/2208.08407/) (2022)
- [Two-in-One Depth: Bridging the Gap Between Monocular and Binocular Self-supervised Depth Estimation](https://mlsanity.com/paper/2309.00933/) (2023)
- [Vision-Based Environmental Perception for Autonomous Driving](https://mlsanity.com/paper/2212.11453/) (2022)
- [ViM-Disparity: Bridging the Gap of Speed, Accuracy and Memory for Disparity Map Generation](https://mlsanity.com/paper/2412.16745/) (2024)
- [A Compacted Structure for Cross-domain learning on Monocular Depth and Flow Estimation](https://mlsanity.com/paper/2208.11993/) (2022)
- [USAM-Net: A U-Net-based Network for Improved Stereo Correspondence and Scene Depth Estimation using Features from a Pre-trained Image Segmentation network](https://mlsanity.com/paper/2503.14950/) (2025)
- [M3Depth: Wavelet-Enhanced Depth Estimation on Mars via Mutual Boosting of Dual-Modal Data](https://mlsanity.com/paper/2505.14159/) (2025)
- [SatDreamer360: Multiview-Consistent Generation of Ground-Level Scenes from Satellite Imagery](https://mlsanity.com/paper/2506.00600/) (2025)
- [CCNeXt: An Effective Self-Supervised Stereo Depth Estimation Approach](https://mlsanity.com/paper/2509.22627/) (2025)
- [HOT-POT: Optimal Transport for Sparse Stereo Matching](https://mlsanity.com/paper/2601.12423/) (2026)
- [BINO: Encoder Centric Self Supervised Stereo With Native Pair Input](https://mlsanity.com/paper/2603.27904/) (2026)

## Abstract (arXiv)

Depth estimation from a stereo image pair has become one of the most explored applications in computer vision, with most of the previous methods relying on fully supervised learning settings. However, due to the difficulty in acquiring accurate and scalable ground truth data, the training of fully supervised methods is challenging. As an alternative, self-supervised methods are becoming more popular to mitigate this challenge. In this paper, we introduce the H-Net, a deep-learning framework for unsupervised stereo depth estimation that leverages epipolar geometry to refine stereo matching. For the first time, a Siamese autoencoder architecture is used for depth estimation which allows mutual information between the rectified stereo images to be extracted. To enforce the epipolar constraint, the mutual epipolar attention mechanism has been designed which gives more emphasis to correspondences of features which lie on the same epipolar line while learning mutual information between the input stereo pair. Stereo correspondences are further enhanced by incorporating semantic information to the proposed attention mechanism. More specifically, the optimal transport algorithm is used to suppress attention and eliminate outliers in areas not visible in both cameras. Extensive experiments on KITTI2015 and Cityscapes show that our method outperforms the state-ofthe-art unsupervised stereo depth estimation methods while closing the gap with the fully supervised approaches.