# FSNet: Redesign Self-Supervised MonoDepth for Full-Scale Depth Prediction for Autonomous Driving

arXiv: https://arxiv.org/abs/2304.10719 · HTML: https://mlsanity.com/paper/2304.10719/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 51 references that are in the map.
- First posted on arXiv 21 Apr 2023.

## Builds on (most cited first, 30 of 30)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Depth Map Prediction from a Single Image using a Multi-Scale Deep Network](https://mlsanity.com/paper/1406.2283/) (2014)
- [ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial and Multi-Map SLAM](https://mlsanity.com/paper/2007.11898/) (2020)
- [Deep Ordinal Regression Network for Monocular Depth Estimation](https://mlsanity.com/paper/1806.02446/) (2018)
- [KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3D](https://mlsanity.com/paper/2109.13410/) (2021)
- [Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillation](https://mlsanity.com/paper/1905.08094/) (2019)
- [VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator](https://mlsanity.com/paper/1708.03852/) (2017)
- [Unsupervised Scale-consistent Depth and Ego-motion Learning from Monocular Video](https://mlsanity.com/paper/1908.10553/) (2019)
- [Competitive Collaboration: Joint Unsupervised Learning of Depth, Camera Motion, Optical Flow and Motion Segmentation](https://mlsanity.com/paper/1805.09806/) (2018)
- [D3VO: Deep Depth, Deep Pose and Deep Uncertainty for Monocular Visual Odometry](https://mlsanity.com/paper/2003.01060/) (2020)
- [The Temporal Opportunist: Self-Supervised Multi-Frame Monocular Depth](https://mlsanity.com/paper/2104.14540/) (2021)
- [MonoPair: Monocular 3D Object Detection Using Pairwise Spatial Relationships](https://mlsanity.com/paper/2003.00504/) (2020)
- [Regularizing Class-wise Predictions via Self-knowledge Distillation](https://mlsanity.com/paper/2003.13964/) (2020)
- [Self-Supervised Monocular Depth Hints](https://mlsanity.com/paper/1909.09051/) (2019)
- [Learning by Analogy: Reliable Supervision from Transformations for Unsupervised Optical Flow Estimation](https://mlsanity.com/paper/2003.13045/) (2020)
- [Depth Estimation via Affinity Learned with Convolutional Spatial Propagation Network](https://mlsanity.com/paper/1808.00150/) (2018)
- [In Defense of Classical Image Processing: Fast Depth Completion on the CPU](https://mlsanity.com/paper/1802.00036/) (2018)
- [Unsupervised Depth Completion from Visual Inertial Odometry](https://mlsanity.com/paper/1905.08616/) (2019)
- [SurroundDepth: Entangling Surrounding Views for Self-Supervised Multi-Camera Depth Estimation](https://mlsanity.com/paper/2204.03636/) (2022)
- [MonoDistill: Learning Spatial Features for Monocular 3D Object Detection](https://mlsanity.com/paper/2201.10830/) (2022)
- [MonoRec: Semi-Supervised Dense Reconstruction in Dynamic Environments from a Single Moving Camera](https://mlsanity.com/paper/2011.11814/) (2020)
- [Ensemble Knowledge Distillation for Learning Improved and Efficient Networks](https://mlsanity.com/paper/1909.08097/) (2019)
- [Full Surround Monodepth from Multiple Cameras](https://mlsanity.com/paper/2104.00152/) (2021)
- [Toward Hierarchical Self-Supervised Monocular Absolute Depth Estimation for Autonomous Driving Applications](https://mlsanity.com/paper/2004.05560/) (2020)
- [Single Image Depth Prediction with Wavelet Decomposition](https://mlsanity.com/paper/2106.02022/) (2021)
- [GVINS: Tightly Coupled GNSS-Visual-Inertial Fusion for Smooth and Consistent State Estimation](https://mlsanity.com/paper/2103.07899/) (2021)
- [Knowledge distillation via adaptive instance normalization](https://mlsanity.com/paper/2003.04289/) (2020)
- [Deterministic Guided LiDAR Depth Map Completion](https://mlsanity.com/paper/2106.07256/) (2021)

## Built on by (most cited first, 2 of 2)

- [FisheyeDepth: A Real Scale Self-Supervised Depth Estimation Model for Fisheye Camera](https://mlsanity.com/paper/2409.15054/) (2024)
- [InVDriver: Intra-Instance Aware Vectorized Query-Based Autonomous Driving Transformer](https://mlsanity.com/paper/2502.17949/) (2025)

## Abstract (arXiv)

Predicting accurate depth with monocular images is important for low-cost robotic applications and autonomous driving. This study proposes a comprehensive self-supervised framework for accurate scale-aware depth prediction on autonomous driving scenes utilizing inter-frame poses obtained from inertial measurements. In particular, we introduce a Full-Scale depth prediction network named FSNet. FSNet contains four important improvements over existing self-supervised models: (1) a multichannel output representation for stable training of depth prediction in driving scenarios, (2) an optical-flow-based mask designed for dynamic object removal, (3) a self-distillation training strategy to augment the training process, and (4) an optimization-based post-processing algorithm in test time, fusing the results from visual odometry. With this framework, robots and vehicles with only one well-calibrated camera can collect sequences of training image frames and camera poses, and infer accurate 3D depths of the environment without extra labeling work or 3D data. Extensive experiments on the KITTI dataset, KITTI-360 dataset and the nuScenes dataset demonstrate the potential of FSNet. More visualizations are presented in urlhttps://sites.google.com/view/fsnet/home