# TartanVO: A Generalizable Learning-based VO

arXiv: https://arxiv.org/abs/2011.00359 · HTML: https://mlsanity.com/paper/2011.00359/

## Key facts

- Cited by 108 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Oct 2020.

## Built on by (most cited first, 30 of 108)

- [DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Cameras](https://mlsanity.com/paper/2108.10869/) (2021)
- [MonST3R: A Simple Approach for Estimating Geometry in the Presence of Motion](https://mlsanity.com/paper/2410.03825/) (2024)
- [Deep Patch Visual Odometry](https://mlsanity.com/paper/2208.04726/) (2022)
- [Infinite Photorealistic Worlds using Procedural Generation](https://mlsanity.com/paper/2306.09310/) (2023)
- [Infinigen Indoors: Photorealistic Indoor Scenes using Procedural Generation](https://mlsanity.com/paper/2406.11824/) (2024)
- [Toward General-Purpose Robots via Foundation Models: A Survey and Meta-Analysis](https://mlsanity.com/paper/2312.08782/) (2023)
- [RelPose: Predicting Probabilistic Relative Rotation for Single Objects in the Wild](https://mlsanity.com/paper/2208.05963/) (2022)
- [ParticleSfM: Exploiting Dense Point Trajectories for Localizing Moving Cameras in the Wild](https://mlsanity.com/paper/2207.09137/) (2022)
- [Deep Patch Visual SLAM](https://mlsanity.com/paper/2408.01654/) (2024)
- [FlowMap: High-Quality Camera Poses, Intrinsics, and Depth via Gradient Descent](https://mlsanity.com/paper/2404.15259/) (2024)
- [LEAP-VO: Long-term Effective Any Point Tracking for Visual Odometry](https://mlsanity.com/paper/2401.01887/) (2024)
- [Dense RGB SLAM with Neural Implicit Maps](https://mlsanity.com/paper/2301.08930/) (2023)
- [The 8-Point Algorithm as an Inductive Bias for Relative Pose Prediction by ViTs](https://mlsanity.com/paper/2208.08988/) (2022)
- [Deep Event Visual Odometry](https://mlsanity.com/paper/2312.09800/) (2023)
- [DiffPoseNet: Direct Differentiable Camera Pose Estimation](https://mlsanity.com/paper/2203.11174/) (2022)
- [DytanVO: Joint Refinement of Visual Odometry and Motion Segmentation in Dynamic Environments](https://mlsanity.com/paper/2209.08430/) (2022)
- [How Does It Feel? Self-Supervised Costmap Learning for Off-Road Vehicle Traversability](https://mlsanity.com/paper/2209.10788/) (2022)
- [Dynamic Camera Poses and Where to Find Them](https://mlsanity.com/paper/2504.17788/) (2025)
- [PVO: Panoptic Visual Odometry](https://mlsanity.com/paper/2207.01610/) (2022)
- [PlaneFormers: From Sparse View Planes to 3D Reconstruction](https://mlsanity.com/paper/2208.04307/) (2022)
- [DVLO: Deep Visual-LiDAR Odometry with Local-to-Global Feature Fusion and Bi-Directional Structure Alignment](https://mlsanity.com/paper/2403.18274/) (2024)
- [Back on Track: Bundle Adjustment for Dynamic Scene Reconstruction](https://mlsanity.com/paper/2504.14516/) (2025)
- [Perception-as-Control: Fine-grained Controllable Image Animation with 3D-aware Motion Representation](https://mlsanity.com/paper/2501.05020/) (2025)
- [Deep Learning for Camera Calibration and Beyond: A Survey](https://mlsanity.com/paper/2303.10559/) (2023)
- [MAC-VO: Metrics-aware Covariance for Learning-based Stereo Visual Odometry](https://mlsanity.com/paper/2409.09479/) (2024)
- [XVO: Generalized Visual Odometry via Cross-Modal Self-Training](https://mlsanity.com/paper/2309.16772/) (2023)
- [Learning a Depth Covariance Function](https://mlsanity.com/paper/2303.12157/) (2023)
- [Imperative Learning: A Self-supervised Neuro-Symbolic Learning Framework for Robot Autonomy](https://mlsanity.com/paper/2406.16087/) (2024)
- [iSLAM: Imperative SLAM](https://mlsanity.com/paper/2306.07894/) (2023)
- [Fast Encoder-Based 3D from Casual Videos via Point Track Processing](https://mlsanity.com/paper/2404.07097/) (2024)

## Abstract (arXiv)

We present the first learning-based visual odometry (VO) model, which generalizes to multiple datasets and real-world scenarios and outperforms geometry-based methods in challenging scenes. We achieve this by leveraging the SLAM dataset TartanAir, which provides a large amount of diverse synthetic data in challenging environments. Furthermore, to make our VO model generalize across datasets, we propose an up-to-scale loss function and incorporate the camera intrinsic parameters into the model. Experiments show that a single model, TartanVO, trained only on synthetic data, without any finetuning, can be generalized to real-world datasets such as KITTI and EuRoC, demonstrating significant advantages over the geometry-based methods on challenging trajectories. Our code is available at https://github.com/castacks/tartanvo.