# Self-Improving Visual Odometry

arXiv: https://arxiv.org/abs/1812.03245 · HTML: https://mlsanity.com/paper/1812.03245/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Dec 2018.

## Built on by (most cited first, 6 of 6)

- [Deep Event Visual Odometry](https://mlsanity.com/paper/2312.09800/) (2023)
- [LIFT-SLAM: a deep-learning feature-based monocular visual SLAM method](https://mlsanity.com/paper/2104.00099/) (2021)
- [Advances in Inference and Representation for Simultaneous Localization and Mapping](https://mlsanity.com/paper/2103.05041/) (2021)
- [A comparative evaluation of learned feature descriptors on hybrid monocular visual SLAM methods](https://mlsanity.com/paper/2104.00085/) (2021)
- [Learning Visual Information Utility with PIXER](https://mlsanity.com/paper/2409.13151/) (2024)
- [SuperPoint-E: local features for 3D reconstruction via tracking adaptation in endoscopy](https://mlsanity.com/paper/2602.04108/) (2026)

## Abstract (arXiv)

We propose a self-supervised learning framework that uses unlabeled monocular video sequences to generate large-scale supervision for training a Visual Odometry (VO) frontend, a network which computes pointwise data associations across images. Our self-improving method enables a VO frontend to learn over time, unlike other VO and SLAM systems which require time-consuming hand-tuning or expensive data collection to adapt to new environments. Our proposed frontend operates on monocular images and consists of a single multi-task convolutional neural network which outputs 2D keypoints locations, keypoint descriptors, and a novel point stability score. We use the output of VO to create a self-supervised dataset of point correspondences to retrain the frontend. When trained using VO at scale on 2.5 million monocular images from ScanNet, the stability classifier automatically discovers a ranking for keypoints that are not likely to help in VO, such as t-junctions across depth discontinuities, features on shadows and highlights, and dynamic objects like people. The resulting frontend outperforms both traditional methods (SIFT, ORB, AKAZE) and deep learning methods (SuperPoint and LF-Net) in a 3D-to-2D pose estimation task on ScanNet.