# Learning Monocular Visual Odometry via Self-Supervised Long-Term Modeling

arXiv: https://arxiv.org/abs/2007.10983 · HTML: https://mlsanity.com/paper/2007.10983/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Jul 2020.

## Built on by (most cited first, 30 of 39)

- [Unsupervised Scale-consistent Depth Learning from Video](https://mlsanity.com/paper/2105.11610/) (2021)
- [Monocular 3D Object Detection: An Extrinsic Parameter Free Approach](https://mlsanity.com/paper/2106.15796/) (2021)
- [Fine-grained Semantics-aware Representation Enhancement for Self-supervised Monocular Depth Estimation](https://mlsanity.com/paper/2108.08829/) (2021)
- [Deep Two-View Structure-from-Motion Revisited](https://mlsanity.com/paper/2104.00556/) (2021)
- [MonoIndoor: Towards Good Practice of Self-Supervised Monocular Depth Estimation for Indoor Environments](https://mlsanity.com/paper/2107.12429/) (2021)
- [PlaneMVS: 3D Plane Reconstruction from Multi-View Stereo](https://mlsanity.com/paper/2203.12082/) (2022)
- [Efficient Deep Visual and Inertial Odometry with Adaptive Visual Modality Selection](https://mlsanity.com/paper/2205.06187/) (2022)
- [Sound Localization from Motion: Jointly Learning Sound Direction and Camera Rotation](https://mlsanity.com/paper/2303.11329/) (2023)
- [Adversarial Training of Self-supervised Monocular Depth Estimation against Physical-World Attacks](https://mlsanity.com/paper/2301.13487/) (2023)
- [DualRefine: Self-Supervised Depth and Pose Estimation Through Iterative Epipolar Sampling and Refinement Toward Equilibrium](https://mlsanity.com/paper/2304.03560/) (2023)
- [Deep Learning for Visual Localization and Mapping: A Survey](https://mlsanity.com/paper/2308.14039/) (2023)
- [MonoIndoor++:Towards Better Practice of Self-Supervised Monocular Depth Estimation for Indoor Environments](https://mlsanity.com/paper/2207.08951/) (2022)
- [JPerceiver: Joint Perception Network for Depth, Pose and Layout Estimation in Driving Scenes](https://mlsanity.com/paper/2207.07895/) (2022)
- [GeoRefine: Self-Supervised Online Depth Refinement for Accurate Dense Mapping](https://mlsanity.com/paper/2205.01656/) (2022)
- [Scene-agnostic Pose Regression for Visual Localization](https://mlsanity.com/paper/2503.19543/) (2025)
- [ColDE: A Depth Estimation Framework for Colonoscopy Reconstruction](https://mlsanity.com/paper/2111.10371/) (2021)
- [MotionHint: Self-Supervised Monocular Visual Odometry with Motion Constraints](https://mlsanity.com/paper/2109.06768/) (2021)
- [Self-Supervised Ego-Motion Estimation Based on Multi-Layer Fusion of RGB and Inferred Depth](https://mlsanity.com/paper/2203.01557/) (2022)
- [Learning Generalized Visual Odometry Using Position-Aware Optical Flow and Geometric Bundle Adjustment](https://mlsanity.com/paper/2111.11141/) (2021)
- [Self-supervised Adversarial Training of Monocular Depth Estimation against Physical-World Attacks](https://mlsanity.com/paper/2406.05857/) (2024)
- [Globally Consistent Video Depth and Pose Estimation with Efficient Test-Time Training](https://mlsanity.com/paper/2208.02709/) (2022)
- [TransFusionOdom: Interpretable Transformer-based LiDAR-Inertial Fusion Odometry Estimation](https://mlsanity.com/paper/2304.07728/) (2023)
- [EMR-MSF: Self-Supervised Recurrent Monocular Scene Flow Exploiting Ego-Motion Rigidity](https://mlsanity.com/paper/2309.01296/) (2023)
- [FisheyeDistill: Self-Supervised Monocular Depth Estimation with Ordinal Distillation for Fisheye Cameras](https://mlsanity.com/paper/2205.02930/) (2022)
- [DeepMLE: A Robust Deep Maximum Likelihood Estimator for Two-view Structure from Motion](https://mlsanity.com/paper/2210.05517/) (2022)
- [Uncertainty-Driven Dense Two-View Structure from Motion](https://mlsanity.com/paper/2302.00523/) (2023)
- [On the Coupling of Depth and Egomotion Networks for Self-Supervised Structure from Motion](https://mlsanity.com/paper/2106.04007/) (2021)
- [Semantics-Depth-Symbiosis: Deeply Coupled Semi-Supervised Learning of Semantics and Depth](https://mlsanity.com/paper/2206.10562/) (2022)
- [Pseudo-LiDAR for Visual Odometry](https://mlsanity.com/paper/2209.01567/) (2022)
- [ROIFormer: Semantic-Aware Region of Interest Transformer for Efficient Self-Supervised Monocular Depth Estimation](https://mlsanity.com/paper/2212.05729/) (2022)

## Abstract (arXiv)

Monocular visual odometry (VO) suffers severely from error accumulation during frame-to-frame pose estimation. In this paper, we present a self-supervised learning method for VO with special consideration for consistency over longer sequences. To this end, we model the long-term dependency in pose prediction using a pose network that features a two-layer convolutional LSTM module. We train the networks with purely self-supervised losses, including a cycle consistency loss that mimics the loop closure module in geometric VO. Inspired by prior geometric systems, we allow the networks to see beyond a small temporal window during training, through a novel a loss that incorporates temporally distant (e.g., O(100)) frames. Given GPU memory constraints, we propose a stage-wise training mechanism, where the first stage operates in a local time window and the second stage refines the poses with a "global" loss given the first stage features. We demonstrate competitive results on several standard VO datasets, including KITTI and TUM RGB-D.