# Semantic-Aware Guided Drone Exploration for Language-Conditioned 3D Indoor Mapping

arXiv: https://arxiv.org/abs/2605.23160 · HTML: https://mlsanity.com/paper/2605.23160/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 30 of its 35 references that are in the map.
- First posted on arXiv 22 May 2026.

## Builds on (most cited first, 30 of 30)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022)
- [Sigmoid Loss for Language Image Pre-Training](https://mlsanity.com/paper/2303.15343/) (2023)
- [Matterport3D: Learning from RGB-D Data in Indoor Environments](https://mlsanity.com/paper/1709.06158/) (2017)
- [Habitat: A Platform for Embodied AI Research](https://mlsanity.com/paper/1904.01201/) (2019)
- [Object Goal Navigation using Goal-Oriented Semantic Exploration](https://mlsanity.com/paper/2007.00643/) (2020)
- [Visual Language Maps for Robot Navigation](https://mlsanity.com/paper/2210.05714/) (2022)
- [ScanQA: 3D Question Answering for Spatial Scene Understanding](https://mlsanity.com/paper/2112.10482/) (2021)
- [ConceptFusion: Open-set Multimodal 3D Mapping](https://mlsanity.com/paper/2302.07241/) (2023)
- [Auxiliary Tasks and Exploration Enable ObjectNav](https://mlsanity.com/paper/2104.04112/) (2021)
- [FUEL: Fast UAV Exploration using Incremental Frontier Structure and Hierarchical Planning](https://mlsanity.com/paper/2010.11561/) (2020)
- [3D Question Answering](https://mlsanity.com/paper/2112.08359/) (2021)
- [Spatiality-guided Transformer for 3D Dense Captioning on Point Clouds](https://mlsanity.com/paper/2204.10688/) (2022)
- [End-to-End Navigation with Vision Language Models: Transforming Spatial Reasoning into Question-Answering](https://mlsanity.com/paper/2411.05755/) (2024)
- [Fast Frontier-based Information-driven Autonomous Exploration with an MAV](https://mlsanity.com/paper/2002.04440/) (2020)
- [SEEK: Semantic Reasoning for Object Goal Navigation in Real World Inspection Tasks](https://mlsanity.com/paper/2405.09822/) (2024)
- [RT-GuIDE: Real-Time Gaussian Splatting for Information-Driven Exploration](https://mlsanity.com/paper/2409.18122/) (2024)
- [A Multi-Resolution Frontier-Based Planner for Autonomous 3D Exploration](https://mlsanity.com/paper/2011.02182/) (2020)
- [Semantic Mapping in Indoor Embodied AI -- A Survey on Advances, Challenges, and Future Directions](https://mlsanity.com/paper/2501.05750/) (2025)
- [FALCON: Fast Autonomous Aerial Exploration using Coverage Path Guidance](https://mlsanity.com/paper/2407.00577/) (2024)
- [RAVEN: Resilient Aerial Navigation via Open-Set Semantic Memory and Behavior Adaptation](https://mlsanity.com/paper/2509.23563/) (2025)
- [Finding Things in the Unknown: Semantic Object-Centric Exploration with an MAV](https://mlsanity.com/paper/2302.14569/) (2023)
- [FindAnything: Open-Vocabulary and Object-Centric Mapping for Robot Exploration in Any Environment](https://mlsanity.com/paper/2504.08603/) (2025)
- [VISTA: Open-Vocabulary, Task-Relevant Robot Exploration with Online Semantic Gaussian Splatting](https://mlsanity.com/paper/2507.01125/) (2025)
- [EDEN: Efficient Dual-Layer Exploration Planning for Fast UAV Autonomous Exploration in Large 3-D Environments](https://mlsanity.com/paper/2506.05106/) (2025)

## Abstract (arXiv)

We present Semantic-Aware Guided Exploration, SAGE, a system for open-vocabulary exploration in unknown 3D indoor environments that preserves coverage-oriented behavior while allowing semantic cues to reprioritize frontier selection. Building on the FALCON volumetric explorer, SAGE integrates Contrastive Language-Image Pre-training (CLIP) via four key components: object-centric embedding storage, a temporal cache that projects recent observations onto the free-unknown boundary, object frontiers for high-similarity detections, and a unified semantic-geometric planning cost. This cost function bounds semantic reweighting influence, ensuring frontiers are prioritized without sacrificing total coverage. In Matterport3D-based simulations, SAGE outperforms FALCON and a semantic-only ablation in object discovery across map-query pairs. Compared to Finding Things in the Unknown (FTU), SAGE completes exploration 9.0 to 25.9 times faster across the nine shared map-query pairs, achieving a mean speedup of 13.7. Furthermore, SAGE achieves substantially higher volumetric throughput than FTU. Finally, we deploy SAGE in five real-world flights in two environments on a Modal AI Starling 2 quadrotor with onboard sensing and planning, and offboard CLIP inference. Comparing SAGE and FALCON, we find that while FALCON results in faster exploration and shorter mapping trajectories, SAGE outperforms FALCON in terms of object discovery.