# Transformer-Based Model for Monocular Visual Odometry: A Video Understanding Approach

arXiv: https://arxiv.org/abs/2305.06121 · HTML: https://mlsanity.com/paper/2305.06121/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 47 references that are in the map.
- First posted on arXiv 10 May 2023.

## Builds on (most cited first, 25 of 25)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [SlowFast Networks for Video Recognition](https://mlsanity.com/paper/1812.03982/) (2018)
- [Is Space-Time Attention All You Need for Video Understanding?](https://mlsanity.com/paper/2102.05095/) (2021)
- [ViViT: A Video Vision Transformer](https://mlsanity.com/paper/2103.15691/) (2021)
- [SuperGlue: Learning Feature Matching with Graph Neural Networks](https://mlsanity.com/paper/1911.11763/) (2019)
- [FlowNet: Learning Optical Flow with Convolutional Networks](https://mlsanity.com/paper/1504.06852/) (2015)
- [ORB-SLAM: a Versatile and Accurate Monocular SLAM System](https://mlsanity.com/paper/1502.00956/) (2015)
- [ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial and Multi-Map SLAM](https://mlsanity.com/paper/2007.11898/) (2020)
- [ORB-SLAM2: an Open-Source SLAM System for Monocular, Stereo and RGB-D Cameras](https://mlsanity.com/paper/1610.06475/) (2016)
- [Quantifying Attention Flow in Transformers](https://mlsanity.com/paper/2005.00928/) (2020)
- [X3D: Expanding Architectures for Efficient Video Recognition](https://mlsanity.com/paper/2004.04730/) (2020)
- [DeepVO: Towards End-to-End Visual Odometry with Deep Recurrent Convolutional Neural Networks](https://mlsanity.com/paper/1709.08429/) (2017)
- [UnDeepVO: Monocular Visual Odometry through Unsupervised Deep Learning](https://mlsanity.com/paper/1709.06841/) (2017)
- [Visual Odometry Revisited: What Should Be Learnt?](https://mlsanity.com/paper/1909.09803/) (2019)
- [SelfVIO: Self-Supervised Deep Monocular Visual-Inertial Odometry and Depth Estimation](https://mlsanity.com/paper/1911.09968/) (2019)
- [DF-VO: What Should Be Learnt for Visual Odometry?](https://mlsanity.com/paper/2103.00933/) (2021)
- [LIFT-SLAM: a deep-learning feature-based monocular visual SLAM method](https://mlsanity.com/paper/2104.00099/) (2021)
- [Approaches, Challenges, and Applications for Deep Visual Odometry: Toward to Complicated and Emerging Areas](https://mlsanity.com/paper/2009.02672/) (2020)
- [MDN-VO: Estimating Visual Odometry with Confidence](https://mlsanity.com/paper/2112.12812/) (2021)
- [Dense Prediction Transformer for Scale Estimation in Monocular Visual Odometry](https://mlsanity.com/paper/2210.01723/) (2022)
- [AFT-VO: Asynchronous Fusion Transformers for Multi-View Visual Odometry Estimation](https://mlsanity.com/paper/2206.12946/) (2022)
- [RAUM-VO: Rotational Adjusted Unsupervised Monocular Visual Odometry](https://mlsanity.com/paper/2203.07162/) (2022)

## Built on by (most cited first, 8 of 8)

- [Scene-agnostic Pose Regression for Visual Localization](https://mlsanity.com/paper/2503.19543/) (2025)
- [BRIGHT-VO: Brightness-Guided Hybrid Transformer for Visual Odometry with Multi-modality Refinement Module](https://mlsanity.com/paper/2501.08659/) (2025)
- [PoseFM: Relative Camera Pose Estimation Through Flow Matching](https://mlsanity.com/paper/2604.22350/) (2026)
- [Motion Consistency Loss for Monocular Visual Odometry with Attention-Based Deep Learning](https://mlsanity.com/paper/2401.10857/) (2024)
- [Geometry-Constrained Monocular Scale Estimation Using Semantic Segmentation for Dynamic Scenes](https://mlsanity.com/paper/2503.04235/) (2025)
- [Monocular Visual Odometry without Calibration or Test-time Optimization](https://mlsanity.com/paper/2510.03348/) (2025)
- [Co-training with Ego-centric Video and Demonstration for Robot Navigation Task](https://mlsanity.com/paper/2606.01951/) (2026)
- [TAPAS: Throughput-adaptive Perception for Autonomous Systems](https://mlsanity.com/paper/2607.17317/) (2026)

## Abstract (arXiv)

Estimating the camera's pose given images from a single camera is a traditional task in mobile robots and autonomous vehicles. This problem is called monocular visual odometry and often relies on geometric approaches that require considerable engineering effort for a specific scenario. Deep learning methods have been shown to be generalizable after proper training and with a large amount of available data. Transformer-based architectures have dominated the state-of-the-art in natural language processing and computer vision tasks, such as image and video understanding. In this work, we deal with the monocular visual odometry as a video understanding task to estimate the 6 degrees of freedom of a camera's pose. We contribute by presenting the TSformer-VO model based on spatio-temporal self-attention mechanisms to extract features from clips and estimate the motions in an end-to-end manner. Our approach achieved competitive state-of-the-art performance compared with geometry-based and deep learning-based methods on the KITTI visual odometry dataset, outperforming the DeepVO implementation highly accepted in the visual odometry community. The code is publicly available at https://github.com/aofrancani/TSformer-VO.