# A General Optimization-based Framework for Local Odometry Estimation with Multiple Sensors

arXiv: https://arxiv.org/abs/1901.03638 · HTML: https://mlsanity.com/paper/1901.03638/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Jan 2019.

## Built on by (most cited first, 30 of 45)

- [Gaussian Splatting SLAM](https://mlsanity.com/paper/2312.06741/) (2023)
- [NeuralRecon: Real-Time Coherent 3D Reconstruction from Monocular Video](https://mlsanity.com/paper/2104.00681/) (2021)
- [TUM-VIE: The TUM Stereo Visual-Inertial Event Dataset](https://mlsanity.com/paper/2108.07329/) (2021)
- [Deep Event Visual Odometry](https://mlsanity.com/paper/2312.09800/) (2023)
- [PlanarRecon: Real-time 3D Plane Detection and Reconstruction from Posed Monocular Videos](https://mlsanity.com/paper/2206.07710/) (2022)
- [PL-EVIO: Robust Monocular Event-based Visual Inertial Odometry with Point and Line Features](https://mlsanity.com/paper/2209.12160/) (2022)
- [FLSea: Underwater Visual-Inertial and Stereo-Vision Forward-Looking Datasets](https://mlsanity.com/paper/2302.12772/) (2023)
- [Pixel-Aligned Recurrent Queries for Multi-View 3D Object Detection](https://mlsanity.com/paper/2310.01401/) (2023)
- [Deep Visual Odometry with Events and Frames](https://mlsanity.com/paper/2309.09947/) (2023)
- [A Safer Vision-based Autonomous Planning System for Quadrotor UAVs with Dynamic Obstacle Trajectory Prediction and Its Application with LLMs](https://mlsanity.com/paper/2311.12893/) (2023)
- [S3E: A Multi-Robot Multimodal Dataset for Collaborative SLAM](https://mlsanity.com/paper/2210.13723/) (2022)
- [DVI-SLAM: A Dual Visual Inertial SLAM Network](https://mlsanity.com/paper/2309.13814/) (2023)
- [RD-VIO: Robust Visual-Inertial Odometry for Mobile Augmented Reality in Dynamic Environments](https://mlsanity.com/paper/2310.15072/) (2023)
- [EVI-SAM: Robust, Real-time, Tightly-coupled Event-Visual-Inertial State Estimation and 3D Dense Mapping](https://mlsanity.com/paper/2312.11911/) (2023)
- [OV2SLAM : A Fully Online and Versatile Visual SLAM for Real-Time Applications](https://mlsanity.com/paper/2102.04060/) (2021)
- [SmartPortraits: Depth Powered Handheld Smartphone Dataset of Human Portraits for State Estimation, Reconstruction and Synthesis](https://mlsanity.com/paper/2204.10211/) (2022)
- [LuSNAR:A Lunar Segmentation, Navigation and Reconstruction Dataset based on Muti-sensor for Autonomous Exploration](https://mlsanity.com/paper/2407.06512/) (2024)
- [Event Camera Meets Mobile Embodied Perception: Abstraction, Algorithm, Acceleration, Application](https://mlsanity.com/paper/2503.22943/) (2025)
- [Event Camera-based Visual Odometry for Dynamic Motion Tracking of a Legged Robot Using Adaptive Time Surface](https://mlsanity.com/paper/2305.08962/) (2023)
- [SchurVINS: Schur Complement-Based Lightweight Visual Inertial Navigation System](https://mlsanity.com/paper/2312.01616/) (2023)
- [DH-PTAM: A Deep Hybrid Stereo Events-Frames Parallel Tracking And Mapping System](https://mlsanity.com/paper/2306.01891/) (2023)
- [On-the-Fly SfM: What you capture is What you get](https://mlsanity.com/paper/2309.11883/) (2023)
- [Visual-Inertial SLAM for Unstructured Outdoor Environments: Benchmarking the Benefits and Computational Costs of Loop Closing](https://mlsanity.com/paper/2408.01716/) (2024)
- [ROVER: A Multi-Season Dataset for Visual SLAM](https://mlsanity.com/paper/2412.02506/) (2024)
- [High-Speed Stereo Visual SLAM for Low-Powered Computing Devices](https://mlsanity.com/paper/2410.04090/) (2024)
- [Benchmarking Visual-Inertial Deep Multimodal Fusion for Relative Pose Regression and Odometry-aided Absolute Pose Regression](https://mlsanity.com/paper/2208.00919/) (2022)
- [Design, Implementation and Evaluation of an External Pose-Tracking System for Underwater Cameras](https://mlsanity.com/paper/2305.04226/) (2023)
- [FlowPilot: Real-Time World-Action Modeling for Agile UAV Navigation](https://mlsanity.com/paper/2608.00635/) (2026)
- [Loop Closure Detection Based on Object-level Spatial Layout and Semantic Consistency](https://mlsanity.com/paper/2304.05146/) (2023)
- [Hyperion -- A fast, versatile symbolic Gaussian Belief Propagation framework for Continuous-Time SLAM](https://mlsanity.com/paper/2407.07074/) (2024)

## Abstract (arXiv)

Nowadays, more and more sensors are equipped on robots to increase robustness and autonomous ability. We have seen various sensor suites equipped on different platforms, such as stereo cameras on ground vehicles, a monocular camera with an IMU (Inertial Measurement Unit) on mobile phones, and stereo cameras with an IMU on aerial robots. Although many algorithms for state estimation have been proposed in the past, they are usually applied to a single sensor or a specific sensor suite. Few of them can be employed with multiple sensor choices. In this paper, we proposed a general optimization-based framework for odometry estimation, which supports multiple sensor sets. Every sensor is treated as a general factor in our framework. Factors which share common state variables are summed together to build the optimization problem. We further demonstrate the generality with visual and inertial sensors, which form three sensor suites (stereo cameras, a monocular camera with an IMU, and stereo cameras with an IMU). We validate the performance of our system on public datasets and through real-world experiments with multiple sensors. Results are compared against other state-of-the-art algorithms. We highlight that our system is a general framework, which can easily fuse various sensors in a pose graph optimization. Our implementations are open sourcefootnotehttps://github.com/HKUST-Aerial-Robotics/VINS-Fusion.