NaVIP: An Image-Centric Indoor Navigation Solution for Visually Impaired People
Not on the map. Why?
Abstract (arXiv)
Indoor navigation is challenging due to the absence of satellite positioning. This challenge is manifold greater for Visually Impaired People (VIPs) who lack the ability to get information from wayfinding signage. Other sensor signals (e.g., Bluetooth and LiDAR) can be used to create turn-by-turn navigation solutions with position updates for users. Unfortunately, these solutions require tags to be installed all around the environment or the use of fairly expensive hardware. Moreover, these solutions require a high degree of manual involvement that raises costs, thus hampering scalability. We propose an image dataset and associated image-centric solution called NaVIP towards visual intelligence that is infrastructure-free and task-scalable, and can assist VIPs in understanding their surroundings. Specifically, we start by curating large-scale phone camera data in a four-floor research building, with 300K images, to lay the foundation for creating an image-centric indoor navigation and exploration solution for inclusiveness. Every image is labelled with precise 6DoF camera poses, details of indoor PoIs, and descriptive captions to assist VIPs. We benchmark on two main aspects: 1) positioning system and 2) exploration support, prioritizing training scalability and real-time inference, to validate the prospect of image-based solution towards indoor navigation. The dataset, code, and model checkpoints are made publicly available at https://github.com/junfish/VIP_Navi.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 84 of its 159 references that are in the map. First posted on arXiv 8 Oct 2024.
Builds on
84 of 159 references.
References · 84
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2023GPT-4 Technical Reportcited by 18,396
- 2015Distilling the Knowledge in a Neural Networkcited by 10,324
- 2023Segment Anythingcited by 8,234
- 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,859
- 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
- 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,046
- 2019Searching for MobileNetV3cited by 1,994
- 2024Depth Anything: Unleashing the Power of Large-Scale Unlabeled Datacited by 1,360
- 2017Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semanticscited by 1,358
- 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,274
- 2020NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collectionscited by 1,232
38 more
- 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,027
- 2018VizWiz Grand Challenge: Answering Visual Questions from Blind Peoplecited by 858
- 2016SPICE: Semantic Propositional Image Caption Evaluationcited by 824
- 2017PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenescited by 775
- 2022OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Frameworkcited by 745
- 2021DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Camerascited by 687
- 2018From Coarse to Fine: Robust Hierarchical Localization at Large Scalecited by 559
- 2021ClipCap: CLIP Prefix for Image Captioningcited by 559
- 2019Object Detection in 20 Years: A Surveycited by 531
- 2018Object Detection with Deep Learning: A Reviewcited by 528
- 2018Deep Learning for Generic Object Detection: A Surveycited by 392
- 2023A Comprehensive Survey on Pretrained Foundation Models: A History from BERT to ChatGPTcited by 368
- 2019DenseFusion: 6D Object Pose Estimation by Iterative Dense Fusioncited by 355
- 2018Speaker-Follower Models for Vision-and-Language Navigationcited by 340
- 2019REVERIE: Remote Embodied Visual Referring Expression in Real Indoor Environmentscited by 333
- 2019Learning to Navigate Unseen Environments: Back Translation with Environmental Dropoutcited by 244
- 2017Geometric Loss Functions for Camera Pose Regression with Deep Learningcited by 224
- 2018A Comprehensive Survey of Deep Learning for Image Captioningcited by 214
- 2021Back to the Feature: Learning Robust Camera Localization from Pixels to Posecited by 199
- 2019Understanding the Limitations of CNN-based Absolute Camera Pose Regressioncited by 177
- 2017Geometry-Aware Learning of Maps for Camera Localizationcited by 161
- 2016Image-based localization using LSTMs for structured feature correlationcited by 136
- 2015Modelling Uncertainty in Deep Learning for Camera Relocalizationcited by 134
- 2021Pathdreamer: A World Model for Indoor Navigationcited by 118
- 2022Vision-and-Language Navigation: A Survey of Tasks, Methods, and Future Directionscited by 117
- 2023Scaling Data Generation in Vision-and-Language Navigationcited by 117
- 2016Show and Tell: Lessons learned from the 2015 MSCOCO Image Captioning Challengecited by 116
- 2021Learning Multi-Scene Absolute Pose Regression with Transformerscited by 108
- 2022LaMAR: Benchmarking Localization and Mapping for Augmented Realitycited by 100
- 2019Hierarchical Scene Coordinate Classification and Regression for Visual Localizationcited by 93
- 2019Single-Stage 6D Object Pose Estimationcited by 88
- 2022DFNet: Enhance Absolute Pose Regression with Direct Feature Matchingcited by 82
- 2017Camera Relocalization by Computing Pairwise Relative Poses Using Convolutional Neural Networkcited by 79
- 2023Bird's-Eye-View Scene Graph for Vision-Language Navigationcited by 72
- 2018Particle Filter Networks with Application to Visual Localizationcited by 65
- 2017Image-based Localization using Hourglass Networkscited by 63
- 2016Learning to Navigate the Energy Landscapecited by 62
- 2021Direct-PoseNet: Absolute Pose Regression with Photometric Consistencycited by 61
50 most cited of 84 All 84 (JSON)
Built on by
Citing papers per year: 2025: 2.
2 papers.