mlsanityData to 14 Oct 2026

RayFronts: Open-Set Semantic Ray Frontiers for Online Scene Understanding and Exploration

arXiv 2504.06994 ↗First posted 9 Apr 2025cs.RO cs.AI cs.CVcited by 23

Not on the map. Why?

Abstract (arXiv)

Open-set semantic mapping is crucial for open-world robots. Current mapping approaches either are limited by the depth range or only map beyond-range entities in constrained settings, where overall they fail to combine within-range and beyond-range observations. Furthermore, these methods make a trade-off between fine-grained semantics and efficiency. We introduce RayFronts, a unified representation that enables both dense and beyond-range efficient semantic mapping. RayFronts encodes task-agnostic open-set semantics to both in-range voxels and beyond-range rays encoded at map boundaries, empowering the robot to reduce search volumes significantly and make informed decisions both within & beyond sensory range, while running at 8.84 Hz on an Orin AGX. Benchmarking the within-range semantics shows that RayFronts's fine-grained image encoding provides 1.34x zero-shot 3D semantic segmentation performance while improving throughput by 16.5x. Traditionally, online mapping performance is entangled with other system components, complicating evaluation. We propose a planner-agnostic evaluation framework that captures the utility for online beyond-range search and exploration, and show RayFronts reduces search volume 2.2x more efficiently than the closest online baselines.

Key facts. Cited by 23 papers in the map (data through 14 Oct 2026). Builds on 37 of its 41 references that are in the map. First posted on arXiv 9 Apr 2025.

Builds on

37 of 41 references.

References · 37

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2023Segment Anythingcited by 8,234
  4. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  5. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  6. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  7. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,117
  8. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
  9. 2019The Replica Dataset: A Digital Replica of Indoor Spacescited by 904
  10. 2021Extract Free Dense Labels from CLIPcited by 583
  11. 2022OpenScene: 3D Scene Understanding with Open Vocabulariescited by 559
  12. 2020TartanAir: A Dataset to Push the Limits of Visual SLAMcited by 506
25 more
  1. 2023ConceptGraphs: Open-Vocabulary 3D Scene Graphs for Perception and Planningcited by 365
  2. 2023ConceptFusion: Open-set Multimodal 3D Mappingcited by 304
  3. 2023VLFM: Vision-Language Frontier Maps for Zero-Shot Semantic Navigationcited by 205
  4. 2024Hierarchical Open-Vocabulary 3D Scene Graphs for Language-Grounded Robot Navigationcited by 196
  5. 2022Hydra: A Real-time Spatial Perception System for 3D Scene Graph Construction and Optimizationcited by 178
  6. 2023AM-RADIO: Agglomerative Vision Foundation Model -- Reduce All Domains Into Onecited by 169
  7. 2023Towards Open Vocabulary Learning: A Surveycited by 166
  8. 2023AnyLoc: Towards Universal Visual Place Recognitioncited by 163
  9. 2022Open-vocabulary Queryable Scene Representations for Real World Planningcited by 160
  10. 2023Toward General-Purpose Robots via Foundation Models: A Survey and Meta-Analysiscited by 103
  11. 2018Fusion++: Volumetric Object-Level SLAMcited by 98
  12. 2018MaskFusion: Real-Time Recognition, Tracking and Reconstruction of Multiple Moving Objectscited by 95
  13. 2024Clio: Real-time Task-Driven Open-Set 3D Scene Graphscited by 81
  14. 2019Volumetric Instance-Aware Semantic Mapping and 3D Object Discoverycited by 80
  15. 2024Pay Attention to Your Neighbours: Training-Free Open-Vocabulary Semantic Segmentationcited by 77
  16. 2024Language-Grounded Dynamic Scene Graphs for Interactive Object Search with Mobile Manipulationcited by 70
  17. 2018MID-Fusion: Octree-based Object-Level Multi-Instance Dynamic SLAMcited by 65
  18. 2023How To Not Train Your Dragon: Training-free Embodied Object Goal Navigation with Semantic Frontierscited by 54
  19. 2024Embodied-RAG: General Non-parametric Embodied Memory for Retrieval and Generationcited by 44
  20. 2024Harnessing Vision Foundation Models for High-Performance, Training-Free Open Vocabulary Segmentationcited by 39
  21. 2023Language-EXtended Indoor SLAM (LEXIS): A Versatile System for Real-time Visual Scene Understandingcited by 15
  22. 2024Map It Anywhere (MIA): Empowering Bird's Eye View Mapping using Large-scale Public Datacited by 5
  23. 2023Multi-Robot Multi-Room Exploration with Geometric Cue Extraction and Circular Decompositioncited by 4
  24. 2022Efficient Global Occupancy Mapping for Mobile Robots using OpenVDBcited by 2
  25. 2024Embedding Pose Graph, Enabling 3D Foundation Model Capabilities with a Compact Representationcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 19.

23 papers.

Sort:

By area:

  1. 2025MapAnything: Universal Feed-Forward Metric 3D Reconstructioncited by 314
  2. 2026C-RADIOv4 (Tech Report)cited by 12
  3. 2025RAVEN: Resilient Aerial Navigation via Open-Set Semantic Memory and Behavior Adaptationcited by 10
  4. 2025FindAnything: Open-Vocabulary and Object-Centric Mapping for Robot Exploration in Any Environmentcited by 8
  5. 2025RADSeg: Unleashing Parameter and Compute Efficient Zero-Shot Open-Vocabulary Segmentation Using Agglomerative Modelscited by 5
  6. 2026WildOS: Open-Vocabulary Object Search in the Wildcited by 3
  7. 2026Explore Like Humans: Autonomous Exploration with Online SG-Memo Construction for Embodied Agentscited by 3
  8. 2026UniScale: Unified Scale-Aware 3D Reconstruction for Multi-View Understanding via Prior Injection for Robotic Perceptioncited by 2
  9. 2026LESV: Language Embedded Sparse Voxel Fusion for Open-Vocabulary 3D Scene Understandingcited by 1
  10. 2026RADIO-ViPE: Online Tightly Coupled Multi-Modal Fusion for Open-Vocabulary Semantic SLAM in Dynamic Environmentscited by 1
  11. 2026FUS3DMaps: Scalable and Accurate Open-Vocabulary Semantic Mapping by 3D Fusion of Voxel- and Instance-Level Layerscited by 1
  12. 2026LEXI-SG: Monocular 3D Scene Graph Mapping with Room-Guided Feed-Forward Reconstructioncited by 1
11 more
  1. 2026Argus: Metric Panoramic 3D Reconstruction for Indoor Scenescited by 1
  2. 2026Actively Resolving Contextual Uncertainty for Underspecified Tasks in Natural Languagecited by 1
  3. 2026R2F: Repurposing Ray Frontiers for LLM-free Object Navigationcited by 0
  4. 2026ProOOD: Prototype-Guided Out-of-Distribution 3D Occupancy Predictioncited by 0
  5. 2026SearchAD: Large-Scale Rare Image Retrieval Dataset for Autonomous Drivingcited by 0
  6. 2026PLAF: Pixel-wise Language-Aligned Feature Extraction for Efficient 3D Scene Understandingcited by 0
  7. 2026G-DRAGON: Geospatial Reasoning and Dynamic Planning for Retrieval-Augmented Outdoor Navigationcited by 0
  8. 2026Don't Fool Me Twice: Adapting to Adversity in the Wild with Experience-Driven Reasoningcited by 0
  9. 2026TARIC: Memory-Augmented Traversability-Aware Outdoor VLN under Interrupted Semantic Cuescited by 0
  10. 2026Beyond Frontiers: Scene-Anomaly Guided Autonomous Explorationcited by 0
  11. 2026Multi-Scale Semantic Mapping in Urban Environments via Observation Calibration and Policy Dependence Regularizationcited by 0