# STAG-VIO: Stabilized Prompt-to-Geometry Interface for Robust Dynamic Visual--Inertial Odometry

arXiv: https://arxiv.org/abs/2411.19289 · HTML: https://mlsanity.com/paper/2411.19289/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 24 of its 33 references that are in the map.
- First posted on arXiv 28 Nov 2024.

## Builds on (most cited first, 24 of 24)

- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023)
- [SAM 2: Segment Anything in Images and Videos](https://mlsanity.com/paper/2408.00714/) (2024)
- [SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation](https://mlsanity.com/paper/1511.00561/) (2015)
- [ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial and Multi-Map SLAM](https://mlsanity.com/paper/2007.11898/) (2020)
- [DETRs Beat YOLOs on Real-time Object Detection](https://mlsanity.com/paper/2304.08069/) (2023)
- [YOLOv11: An Overview of the Key Architectural Enhancements](https://mlsanity.com/paper/2410.17725/) (2024)
- [VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator](https://mlsanity.com/paper/1708.03852/) (2017)
- [Faster Segment Anything: Towards Lightweight SAM for Mobile Applications](https://mlsanity.com/paper/2306.14289/) (2023)
- [FAST-LIO2: Fast Direct LiDAR-inertial Odometry](https://mlsanity.com/paper/2107.06829/) (2021)
- [DynaSLAM: Tracking, Mapping and Inpainting in Dynamic Scenes](https://mlsanity.com/paper/1806.05620/) (2018)
- [DS-SLAM: A Semantic Visual SLAM towards Dynamic Environments](https://mlsanity.com/paper/1809.08379/) (2018)
- [RT-DETRv2: Improved Baseline with Bag-of-Freebies for Real-Time Detection Transformer](https://mlsanity.com/paper/2407.17140/) (2024)
- [A General Optimization-based Framework for Local Odometry Estimation with Multiple Sensors](https://mlsanity.com/paper/1901.03638/) (2019)
- [Are We Ready for Service Robots? The OpenLORIS-Scene Datasets for Lifelong SLAM](https://mlsanity.com/paper/1911.05603/) (2019)
- [MobileSAMv2: Faster Segment Anything to Everything](https://mlsanity.com/paper/2312.09579/) (2023)
- [PVO: Panoptic Visual Odometry](https://mlsanity.com/paper/2207.01610/) (2022)
- [VIODE: A Simulated Dataset to Address the Challenges of Visual-Inertial Odometry in Dynamic Environments](https://mlsanity.com/paper/2102.05965/) (2021)
- [DynaVINS: A Visual-Inertial SLAM for Dynamic Environments](https://mlsanity.com/paper/2208.11500/) (2022)
- [YOLOv5, YOLOv8 and YOLOv10: The Go-To Detectors for Real-time Vision](https://mlsanity.com/paper/2407.02988/) (2024)
- [RD-VIO: Robust Visual-Inertial Odometry for Mobile Augmented Reality in Dynamic Environments](https://mlsanity.com/paper/2310.15072/) (2023)
- [RGB-D Inertial Odometry for a Resource-Restricted Robot in Dynamic Environments](https://mlsanity.com/paper/2304.10987/) (2023)
- [RP-VIO: Robust Plane-based Visual-Inertial Odometry for Dynamic Environments](https://mlsanity.com/paper/2103.10400/) (2021)
- [Computing a many-to-many matching with demands and capacities between two sets using the Hungarian algorithm](https://mlsanity.com/paper/2211.01612/) (2022)

## Abstract (arXiv)

Dynamic visual-inertial odometry (VIO) requires reliable suppression of motion-corrupted measurements, yet prior semantic-assisted approaches depend on category-limited segmenters and degrade under partial occlusion. Promptable foundation segmentation models offer category-agnostic dynamic parsing, but their effectiveness in VIO depends critically on the temporal stability of input prompts---a factor largely overlooked in existing pipelines. When prompts derived from raw detection are jittery or intermittent under occlusion, the resulting masks flicker across frames, destabilizing geometric estimation. We propose STAG-VIO, which formulates dynamic robustness as a perception-to-geometry interface stabilization problem. We introduce uncertainty-adaptive multi-object tracking that models prompt generation as state estimation with bounded noise adaptation, producing temporally coherent box prompts. These stabilized prompts drive a lightweight foundation segmenter whose masks undergo geometry-oriented morphological refinement to establish conservative safety margins. A constraint-budget-aware feature redistribution strategy preserves well-conditioned static measurements when dynamic regions dominate the view. Experiments on VIODE and OpenLORIS-Scene show consistent gains over state-of-the-art baselines. Ablation confirms that prompt stabilization is the single most impactful component, reducing trajectory error by up to 83%.