# FCDSN-DC: An Accurate and Lightweight Convolutional Neural Network for Stereo Estimation with Depth Completion

arXiv: https://arxiv.org/abs/2209.06525 · HTML: https://mlsanity.com/paper/2209.06525/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 42 references that are in the map.
- First posted on arXiv 14 Sep 2022.

## Builds on (most cited first, 26 of 26)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Deformable Convolutional Networks](https://mlsanity.com/paper/1703.06211/) (2017)
- [Deformable ConvNets v2: More Deformable, Better Results](https://mlsanity.com/paper/1811.11168/) (2018)
- [Pyramid Stereo Matching Network](https://mlsanity.com/paper/1803.08669/) (2018)
- [End-to-End Learning of Geometry and Context for Deep Stereo Regression](https://mlsanity.com/paper/1703.04309/) (2017)
- [RAFT-Stereo: Multilevel Recurrent Field Transforms for Stereo Matching](https://mlsanity.com/paper/2109.07547/) (2021)
- [GA-Net: Guided Aggregation Net for End-to-end Stereo Matching](https://mlsanity.com/paper/1904.06587/) (2019)
- [AANet: Adaptive Aggregation Network for Efficient Stereo Matching](https://mlsanity.com/paper/2004.09548/) (2020)
- [Sparse-to-Dense: Depth Prediction from Sparse Depth Samples and a Single Image](https://mlsanity.com/paper/1709.07492/) (2017)
- [Stereo Matching by Training a Convolutional Neural Network to Compare Image Patches](https://mlsanity.com/paper/1510.05970/) (2015)
- [Self-supervised Sparse-to-Dense: Self-supervised Depth Completion from LiDAR and Monocular Camera](https://mlsanity.com/paper/1807.00275/) (2018)
- [Hierarchical Deep Stereo Matching on High-resolution Images](https://mlsanity.com/paper/1912.06704/) (2019)
- [In Defense of Classical Image Processing: Fast Depth Completion on the CPU](https://mlsanity.com/paper/1802.00036/) (2018)
- [Fast End-to-End Trainable Guided Filter](https://mlsanity.com/paper/1803.05619/) (2018)
- [Parallax Attention for Unsupervised Stereo Correspondence Learning](https://mlsanity.com/paper/2009.08250/) (2020)
- [Propagating Confidences through CNNs for Sparse Data Regression](https://mlsanity.com/paper/1805.11913/) (2018)
- [Holopix50k: A Large-Scale In-the-wild Stereo Image Dataset](https://mlsanity.com/paper/2003.11172/) (2020)
- [Reversing the cycle: self-supervised deep stereo through enhanced monocular distillation](https://mlsanity.com/paper/2008.07130/) (2020)
- [An Adaptive Framework for Learning Unsupervised Depth Completion](https://mlsanity.com/paper/2106.03010/) (2021)
- [DispSegNet: Leveraging Semantics for End-to-End Learning of Disparity Estimation from Stereo Imagery](https://mlsanity.com/paper/1809.04734/) (2018)
- [End-to-End Training of Hybrid CNN-CRF Models for Stereo](https://mlsanity.com/paper/1611.10229/) (2016)
- [Belief Propagation Reloaded: Learning BP-Layers for Labeling Problems](https://mlsanity.com/paper/2003.06258/) (2020)
- [Separable Convolutions for Optimizing 3D Stereo Networks](https://mlsanity.com/paper/2108.10216/) (2021)
- [FC-DCNN: A densely connected neural network for stereo estimation](https://mlsanity.com/paper/2010.06950/) (2020)

## Built on by (most cited first, 1 of 1)

- [SAda-Net: A Self-Supervised Adaptive Stereo Estimation CNN For Remote Sensing Image Data](https://mlsanity.com/paper/2410.13500/) (2024)

## Abstract (arXiv)

We propose an accurate and lightweight convolutional neural network for stereo estimation with depth completion. We name this method fully-convolutional deformable similarity network with depth completion (FCDSN-DC). This method extends FC-DCNN by improving the feature extractor, adding a network structure for training highly accurate similarity functions and a network structure for filling inconsistent disparity estimates. The whole method consists of three parts. The first part consists of fully-convolutional densely connected layers that computes expressive features of rectified image pairs. The second part of our network learns highly accurate similarity functions between this learned features. It consists of densely-connected convolution layers with a deformable convolution block at the end to further improve the accuracy of the results. After this step an initial disparity map is created and the left-right consistency check is performed in order to remove inconsistent points. The last part of the network then uses this input together with the corresponding left RGB image in order to train a network that fills in the missing measurements. Consistent depth estimations are gathered around invalid points and are parsed together with the RGB points into a shallow CNN network structure in order to recover the missing values. We evaluate our method on challenging real world indoor and outdoor scenes, in particular Middlebury, KITTI and ETH3D were it produces competitive results. We furthermore show that this method generalizes well and is well suited for many applications without the need of further training. The code of our full framework is available at: https://github.com/thedodo/FCDSN-DC