# DefVINS: Visual-Inertial Odometry for Deformable Scenes

arXiv: https://arxiv.org/abs/2601.00702 · HTML: https://mlsanity.com/paper/2601.00702/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 27 of its 45 references that are in the map.
- First posted on arXiv 2 Jan 2026.

## Builds on (most cited first, 27 of 27)

- [ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial and Multi-Map SLAM](https://mlsanity.com/paper/2007.11898/) (2020)
- [Direct Sparse Odometry](https://mlsanity.com/paper/1607.02565/) (2016)
- [VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator](https://mlsanity.com/paper/1708.03852/) (2017)
- [Past, Present, and Future of Simultaneous Localization And Mapping: Towards the Robust-Perception Age](https://mlsanity.com/paper/1606.05830/) (2016)
- [DynaSLAM: Tracking, Mapping and Inpainting in Dynamic Scenes](https://mlsanity.com/paper/1806.05620/) (2018)
- [On-Manifold Preintegration for Real-Time Visual-Inertial Odometry](https://mlsanity.com/paper/1512.02363/) (2015)
- [VINet: Visual-Inertial Odometry as a Sequence-to-Sequence Learning Problem](https://mlsanity.com/paper/1701.08376/) (2017)
- [DynaSLAM II: Tightly-Coupled Multi-Object Tracking and SLAM](https://mlsanity.com/paper/2010.07820/) (2020)
- [Stereo DSO: Large-Scale Direct Sparse Visual Odometry with Stereo Cameras](https://mlsanity.com/paper/1708.07878/) (2017)
- [Challenges and Outlook in Robotic Manipulation of Deformable Objects](https://mlsanity.com/paper/2105.01767/) (2021)
- [DefSLAM: Tracking and Mapping of Deforming Scenes from Monocular Sequences](https://mlsanity.com/paper/1908.08918/) (2019)
- [Direct Sparse Visual-Inertial Odometry using Dynamic Marginalization](https://mlsanity.com/paper/1804.05625/) (2018)
- [Visual-Inertial Navigation: A Concise Review](https://mlsanity.com/paper/1906.02650/) (2019)
- [Photometric Bundle Adjustment for Vision-Based SLAM](https://mlsanity.com/paper/1608.02026/) (2016)
- [SD-DefSLAM: Semi-Direct Monocular SLAM for Deformable and Intracorporeal Scenes](https://mlsanity.com/paper/2010.09409/) (2020)
- [DOT: Dynamic Object Tracking for Visual SLAM](https://mlsanity.com/paper/2010.00052/) (2020)
- [Benchmarking Egocentric Visual-Inertial SLAM at City Scale](https://mlsanity.com/paper/2509.26639/) (2025)
- [DynaVINS: A Visual-Inertial SLAM for Dynamic Environments](https://mlsanity.com/paper/2208.11500/) (2022)
- [DVI-SLAM: A Dual Visual Inertial SLAM Network](https://mlsanity.com/paper/2309.13814/) (2023)
- [The Drunkard's Odometry: Estimating Camera Motion in Deforming Scenes](https://mlsanity.com/paper/2306.16917/) (2023)
- [Multimotion Visual Odometry (MVO)](https://mlsanity.com/paper/2110.15169/) (2021)
- [DynoSAM: Open-Source Smoothing and Mapping Framework for Dynamic SLAM](https://mlsanity.com/paper/2501.11893/) (2025)
- [Camera Motion Estimation from RGB-D-Inertial Scene Flow](https://mlsanity.com/paper/2404.17251/) (2024)
- [Observability Analysis and Keyframe-Based Filtering for Visual Inertial Odometry with Full Self-Calibration](https://mlsanity.com/paper/2201.04989/) (2022)
- [DynaVINS++: Robust Visual-Inertial State Estimator in Dynamic Environments by Adaptive Truncated Least Squares and Stable State Recovery](https://mlsanity.com/paper/2410.15373/) (2024)
- [SLAM&Render: A Benchmark for the Intersection Between Neural Rendering, Gaussian Splatting and SLAM](https://mlsanity.com/paper/2504.13713/) (2025)
- [GNSS-Inertial State Initialization Using Inter-Epoch Baseline Residuals](https://mlsanity.com/paper/2506.11534/) (2025)

## Abstract (arXiv)

Deformable scenes violate the rigidity assumptions underpinning classical visual--inertial odometry (VIO), often leading to over-fitting to local non-rigid motion or to severe camera pose drift when deformation dominates visual parallax. In this paper, we introduce DefVINS, the first visual-inertial odometry pipeline designed to operate in deformable environments. Our approach models the odometry state by decomposing it into a rigid, IMU-anchored component and a non-rigid scene warp represented by an embedded deformation graph. As a second contribution, we present VIMandala, the first benchmark containing real images and ground-truth camera poses for visual-inertial odometry in deformable scenes. In addition, we augment the synthetic Drunkard's benchmark with simulated inertial measurements to further evaluate our pipeline under controlled conditions. We also provide an observability analysis of the visual-inertial deformable odometry problem, characterizing how inertial measurements constrain camera motion and render otherwise unobservable modes identifiable in the presence of deformation. This analysis motivates the use of IMU anchoring and leads to a conditioning-based activation strategy that avoids ill-posed updates under poor excitation. Experimental results on both the synthetic Drunkard's and our real VIMandala benchmarks show that DefVINS outperforms rigid visual--inertial and non-rigid visual odometry baselines. Our source code and data will be released upon acceptance.