# Visual Odometry Revisited: What Should Be Learnt?

arXiv: https://arxiv.org/abs/1909.09803 · HTML: https://mlsanity.com/paper/1909.09803/

## Key facts

- Cited by 57 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Sep 2019.

## Built on by (most cited first, 30 of 57)

- [Deep Patch Visual Odometry](https://mlsanity.com/paper/2208.04726/) (2022)
- [ParticleSfM: Exploiting Dense Point Trajectories for Localizing Moving Cameras in the Wild](https://mlsanity.com/paper/2207.09137/) (2022)
- [SC-DepthV3: Robust Self-supervised Monocular Depth Estimation for Dynamic Scenes](https://mlsanity.com/paper/2211.03660/) (2022)
- [MonoIndoor: Towards Good Practice of Self-Supervised Monocular Depth Estimation for Indoor Environments](https://mlsanity.com/paper/2107.12429/) (2021)
- [ActiveRMAP: Radiance Field for Active Mapping And Planning](https://mlsanity.com/paper/2211.12656/) (2022)
- [Deep Event Visual Odometry](https://mlsanity.com/paper/2312.09800/) (2023)
- [SAGE: SLAM with Appearance and Geometry Prior for Endoscopy](https://mlsanity.com/paper/2202.09487/) (2022)
- [NARUTO: Neural Active Reconstruction from Uncertain Target Observations](https://mlsanity.com/paper/2402.18771/) (2024)
- [Attention meets Geometry: Geometry Guided Spatial-Temporal Attention for Consistent Self-Supervised Monocular Depth Estimation](https://mlsanity.com/paper/2110.08192/) (2021)
- [DytanVO: Joint Refinement of Visual Odometry and Motion Segmentation in Dynamic Environments](https://mlsanity.com/paper/2209.08430/) (2022)
- [DF-VO: What Should Be Learnt for Visual Odometry?](https://mlsanity.com/paper/2103.00933/) (2021)
- [Spatiotemporal Registration for Event-based Visual Odometry](https://mlsanity.com/paper/2103.05955/) (2021)
- [Towards Scale-Aware, Robust, and Generalizable Unsupervised Monocular Depth Estimation by Integrating IMU Motion Dynamics](https://mlsanity.com/paper/2207.04680/) (2022)
- [Generalizing to the Open World: Deep Visual Odometry with Online Adaptation](https://mlsanity.com/paper/2103.15279/) (2021)
- [XVO: Generalized Visual Odometry via Cross-Modal Self-Training](https://mlsanity.com/paper/2309.16772/) (2023)
- [Deep Learning for Visual Localization and Mapping: A Survey](https://mlsanity.com/paper/2308.14039/) (2023)
- [Deep Visual Odometry with Events and Frames](https://mlsanity.com/paper/2309.09947/) (2023)
- [MonoIndoor++:Towards Better Practice of Self-Supervised Monocular Depth Estimation for Indoor Environments](https://mlsanity.com/paper/2207.08951/) (2022)
- [Deep Online Correction for Monocular Visual Odometry](https://mlsanity.com/paper/2103.10029/) (2021)
- [Towards Scale Consistent Monocular Visual Odometry by Learning from the Virtual World](https://mlsanity.com/paper/2203.05712/) (2022)
- [Transformer-Based Model for Monocular Visual Odometry: A Video Understanding Approach](https://mlsanity.com/paper/2305.06121/) (2023)
- [EMA-VIO: Deep Visual-Inertial Odometry with External Memory Attention](https://mlsanity.com/paper/2209.08490/) (2022)
- [Modality-invariant Visual Odometry for Embodied Vision](https://mlsanity.com/paper/2305.00348/) (2023)
- [Learning Generalized Visual Odometry Using Position-Aware Optical Flow and Geometric Bundle Adjustment](https://mlsanity.com/paper/2111.11141/) (2021)
- [Information-Theoretic Odometry Learning](https://mlsanity.com/paper/2203.05724/) (2022)
- [Fusing Structure from Motion and Simulation-Augmented Pose Regression from Optical Flow for Challenging Indoor Environments](https://mlsanity.com/paper/2304.07250/) (2023)
- [Understanding while Exploring: Semantics-driven Active Mapping](https://mlsanity.com/paper/2506.00225/) (2025)
- [DINO-VO: A Feature-based Visual Odometry Leveraging a Visual Foundation Model](https://mlsanity.com/paper/2507.13145/) (2025)
- [Error Diagnosis of Deep Monocular Depth Estimation Models](https://mlsanity.com/paper/2112.05533/) (2021)
- [Monocular visual simultaneous localization and mapping: (r)evolution from geometry to deep learning-based pipelines](https://mlsanity.com/paper/2503.02955/) (2025)

## Abstract (arXiv)

In this work we present a monocular visual odometry (VO) algorithm which leverages geometry-based methods and deep learning. Most existing VO/SLAM systems with superior performance are based on geometry and have to be carefully designed for different application scenarios. Moreover, most monocular systems suffer from scale-drift issue.Some recent deep learning works learn VO in an end-to-end manner but the performance of these deep systems is still not comparable to geometry-based methods. In this work, we revisit the basics of VO and explore the right way for integrating deep learning with epipolar geometry and Perspective-n-Point (PnP) method. Specifically, we train two convolutional neural networks (CNNs) for estimating single-view depths and two-view optical flows as intermediate outputs. With the deep predictions, we design a simple but robust frame-to-frame VO algorithm (DF-VO) which outperforms pure deep learning-based and geometry-based methods. More importantly, our system does not suffer from the scale-drift issue being aided by a scale consistent single-view depth CNN. Extensive experiments on KITTI dataset shows the robustness of our system and a detailed ablation study shows the effect of different factors in our system.