# Visual-Inertial Mapping with Non-Linear Factor Recovery

arXiv: https://arxiv.org/abs/1904.06504 · HTML: https://mlsanity.com/paper/1904.06504/

## Key facts

- Cited by 50 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Apr 2019.

## Built on by (most cited first, 30 of 50)

- [Kimera: from SLAM to Spatial Perception with 3D Dynamic Scene Graphs](https://mlsanity.com/paper/2101.06894/) (2021)
- [TUM-VIE: The TUM Stereo Visual-Inertial Event Dataset](https://mlsanity.com/paper/2108.07329/) (2021)
- [DM-VIO: Delayed Marginalization Visual-Inertial Odometry](https://mlsanity.com/paper/2201.04114/) (2022)
- [Multi-Event-Camera Depth Estimation and Outlier Rejection by Refocused Events Fusion](https://mlsanity.com/paper/2207.10494/) (2022)
- [Self-Supervised Camera Self-Calibration from Video](https://mlsanity.com/paper/2112.03325/) (2021)
- [From SLAM to Situational Awareness: Challenges and Survey](https://mlsanity.com/paper/2110.00273/) (2021)
- [Learning Correspondence Uncertainty via Differentiable Nonlinear Least Squares](https://mlsanity.com/paper/2305.09527/) (2023)
- [The Monado SLAM Dataset for Egocentric Visual-Inertial Tracking](https://mlsanity.com/paper/2508.00088/) (2025)
- [OV2SLAM : A Fully Online and Versatile Visual SLAM for Real-Time Applications](https://mlsanity.com/paper/2102.04060/) (2021)
- [HybVIO: Pushing the Limits of Real-time Visual-inertial Odometry](https://mlsanity.com/paper/2106.11857/) (2021)
- [The Probabilistic Normal Epipolar Constraint for Frame-To-Frame Rotation Optimization under Uncertain Feature Positions](https://mlsanity.com/paper/2204.02256/) (2022)
- [Square Root Marginalization for Sliding-Window Bundle Adjustment](https://mlsanity.com/paper/2109.02182/) (2021)
- [4Seasons: Benchmarking Visual SLAM and Long-Term Localization for Autonomous Driving in Challenging Conditions](https://mlsanity.com/paper/2301.01147/) (2022)
- [SchurVINS: Schur Complement-Based Lightweight Visual Inertial Navigation System](https://mlsanity.com/paper/2312.01616/) (2023)
- [Comparison of modern open-source visual SLAM approaches](https://mlsanity.com/paper/2108.01654/) (2021)
- [Vision-based Large-scale 3D Semantic Mapping for Autonomous Driving Applications](https://mlsanity.com/paper/2203.01087/) (2022)
- [DH-PTAM: A Deep Hybrid Stereo Events-Frames Parallel Tracking And Mapping System](https://mlsanity.com/paper/2306.01891/) (2023)
- [Continuous Pose for Monocular Cameras in Neural Implicit Representation](https://mlsanity.com/paper/2311.17119/) (2023)
- [Towards Long Term SLAM on Thermal Imagery](https://mlsanity.com/paper/2403.19885/) (2024)
- [Towards Robust Monocular Visual Odometry for Flying Robots on Planetary Missions](https://mlsanity.com/paper/2109.05509/) (2021)
- [IBISCape: A Simulated Benchmark for multi-modal SLAM Systems Evaluation in Large-scale Dynamic Environments](https://mlsanity.com/paper/2206.13455/) (2022)
- [Expansion of Visual Hints for Improved Generalization in Stereo Matching](https://mlsanity.com/paper/2211.00392/) (2022)
- [Gravity-aligned Rotation Averaging with Circular Regression](https://mlsanity.com/paper/2410.12763/) (2024)
- [Visual-Inertial Odometry with Online Calibration of Velocity-Control Based Kinematic Motion Models](https://mlsanity.com/paper/2204.06776/) (2022)
- [Evaluation and comparison of eight popular Lidar and Visual SLAM algorithms](https://mlsanity.com/paper/2208.02063/) (2022)
- [Event-based Stereo Depth Estimation from Ego-motion using Ray Density Fusion](https://mlsanity.com/paper/2210.08927/) (2022)
- [AQUA-SLAM: Tightly-Coupled Underwater Acoustic-Visual-Inertial SLAM with Sensor Calibration](https://mlsanity.com/paper/2503.11420/) (2025)
- [InFlux: A Benchmark for Self-Calibration of Dynamic Intrinsics of Video Cameras](https://mlsanity.com/paper/2510.23589/) (2025)
- [PO-MSCKF: An Efficient Visual-Inertial Odometry by Reconstructing the Multi-State Constrained Kalman Filter with the Pose-only Theory](https://mlsanity.com/paper/2407.01888/) (2024)
- [Hyperion -- A fast, versatile symbolic Gaussian Belief Propagation framework for Continuous-Time SLAM](https://mlsanity.com/paper/2407.07074/) (2024)

## Abstract (arXiv)

Cameras and inertial measurement units are complementary sensors for ego-motion estimation and environment mapping. Their combination makes visual-inertial odometry (VIO) systems more accurate and robust. For globally consistent mapping, however, combining visual and inertial information is not straightforward. To estimate the motion and geometry with a set of images large baselines are required. Because of that, most systems operate on keyframes that have large time intervals between each other. Inertial data on the other hand quickly degrades with the duration of the intervals and after several seconds of integration, it typically contains only little useful information. In this paper, we propose to extract relevant information for visual-inertial mapping from visual-inertial odometry using non-linear factor recovery. We reconstruct a set of non-linear factors that make an optimal approximation of the information on the trajectory accumulated by VIO. To obtain a globally consistent map we combine these factors with loop-closing constraints using bundle adjustment. The VIO factors make the roll and pitch angles of the global map observable, and improve the robustness and the accuracy of the mapping. In experiments on a public benchmark, we demonstrate superior performance of our method over the state-of-the-art approaches.