Robust Localization with Visual-Inertial Odometry Constraints for Markerless Mobile AR
Not on the map. Why?
Abstract (arXiv)
Visual Inertial Odometry (VIO) is an essential component of modern Augmented Reality (AR) applications. However, VIO only tracks the relative pose of the device, leading to drift over time. Absolute pose estimation methods infer the device's absolute pose, but their accuracy depends on the input quality. This paper introduces VIO-APR, a new framework for markerless mobile AR that combines an absolute pose regressor (APR) with a local VIO tracking system. VIO-APR uses VIO to assess the reliability of the APR and the APR to identify and compensate for VIO drift. This feedback loop results in more accurate positioning and more stable AR experiences. To evaluate VIO-APR, we created a dataset that combines camera images with ARKit's VIO system output for six indoor and outdoor scenes of various scales. Over this dataset, VIO-APR improves the median accuracy of popular APR by up to 36% in position and 29% in orientation, increases the percentage of frames in the high (0.25 m, 2circ) accuracy level by up to 112% and reduces the percentage of frames predicted below the low (5 m, 10^circ) accuracy greatly. We implement VIO-APR into a mobile AR application using Unity to demonstrate its capabilities. VIO-APR results in noticeably more accurate localization and a more stable overall experience.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 28 of its 53 references that are in the map. First posted on arXiv 10 Aug 2023.
Builds on
28 of 53 references.
References · 28
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
- 2015ORB-SLAM: a Versatile and Accurate Monocular SLAM Systemcited by 988
- 2016ORB-SLAM2: an Open-Source SLAM System for Monocular, Stereo and RGB-D Camerascited by 806
- 2018From Coarse to Fine: Robust Hierarchical Localization at Large Scalecited by 559
- 2016Direct Sparse Odometrycited by 485
- 2019D2-Net: A Trainable CNN for Joint Detection and Description of Local Featurescited by 326
- 2016Large-Scale Image Retrieval with Attentive Deep Local Featurescited by 238
- 2017Geometric Loss Functions for Camera Pose Regression with Deep Learningcited by 224
- 2019Understanding the Limitations of CNN-based Absolute Camera Pose Regressioncited by 177
- 2017Geometry-Aware Learning of Maps for Camera Localizationcited by 161
- 2015Modelling Uncertainty in Deep Learning for Camera Relocalizationcited by 134
- 2021LENS: Localization enhanced by NeRF synthesiscited by 117
16 more
- 2021Learning Multi-Scene Absolute Pose Regression with Transformerscited by 108
- 2022LaMAR: Benchmarking Localization and Mapping for Augmented Realitycited by 100
- 2022DFNet: Enhance Absolute Pose Regression with Direct Feature Matchingcited by 82
- 2017Image-based Localization using Hourglass Networkscited by 63
- 2021On the Limits of Pseudo Ground Truth in Visual Camera Re-localisationcited by 63
- 2016Learning to Navigate the Energy Landscapecited by 62
- 2018VLocNet++: Deep Multitask Learning for Semantic Visual Localization and Odometrycited by 61
- 2021Direct-PoseNet: Absolute Pose Regression with Photometric Consistencycited by 61
- 2018Deep Auxiliary Learning for Visual Localization and Odometrycited by 59
- 2020Deep Bingham Networks: Dealing with Uncertainty and Ambiguity in Pose Estimationcited by 43
- 2018Project AutoVision: Localization and 3D Scene Perception for an Autonomous Vehicle with a Multi-Camera Systemcited by 34
- 2021CoordiNet: uncertainty-aware pose regressor for reliable vehicle localizationcited by 34
- 2018ADVIO: An authentic dataset for visual-inertial odometrycited by 23
- 20206D Camera Relocalization in Ambiguous Scenes via Continuous Multimodal Inferencecited by 23
- 2023A Probabilistic Framework for Visual Localization in Ambiguous Scenescited by 12
- 2022Improving Feature-based Visual Localization by Geometry-Aided Matchingcited by 8
Built on by
Not cited yet.