mlsanityData to 14 Oct 2026

Finder: Agentic Closed-Loop Object Finding for Embodied Grounding

arXiv 2609.18058 ↗First posted 16 Sep 2026cs.CVcited by 0

On the map: LLM agents and tool use → Multimodal and embodied agents

Abstract (arXiv)

Finding the object referred to by language in a partially observed 3D scene is a core capability for embodied agents. Existing approaches either couple object search with online exploration, which can be costly when relevant observations have already been captured, or query pre-built open-vocabulary maps and scene graphs in a static, one-shot fashion. We present Finder, an agentic closed-loop object-finding primitive for embodied grounding. Instead of treating grounding as passive retrieval from a fixed scene representation, Finder maintains a typed loop state that links query-conditioned planning, scoped evidence gathering, candidate verification, and accept/continue/abort control. When evidence is incomplete or ambiguous, the loop can redirect subsequent perception and comparison rather than simply returning the top retrieved object. On open-vocabulary embodied Object Retrieval in Habitat/HM3D and real-world RGB-D scenes, Finder improves the averaged 1m success rate by 15.75 points over strong baselines. The same primitive also transfers to sequential object grounding and embodied object-centric question answering, improving spatial and temporal localization without changing the inner grounding protocol. Project page: https://finder-vln.github.io.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 35 of its 50 references that are in the map. First posted on arXiv 16 Sep 2026.

Builds on

35 of 50 references.

Closest ancestors · 3

Same direction, newest first

  1. 2026MALLVI: A Multi-Agent Framework for Integrated Generalized Robotics Manipulation18 Feb
  2. 2025Towards Embodied Agentic AI: Review and Classification of LLM- and VLM-Driven Robot Autonomy and Interaction7 Aug
  3. 2023LLM-Grounder: Open-Vocabulary 3D Visual Grounding with Large Language Model as an Agent21 Sep

Other references · 32

  1. 2025SAM 3: Segment Anything with Conceptscited by 906
  2. 2019ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Taskscited by 783
  3. 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 592
  4. 2022OpenScene: 3D Scene Understanding with Open Vocabulariescited by 559
  5. 2023ConceptGraphs: Open-Vocabulary 3D Scene Graphs for Perception and Planningcited by 365
  6. 2024NVILA: Efficient Frontier Visual Language Modelscited by 209
  7. 2021TEACh: Task-driven Embodied Agents that Chatcited by 197
  8. 2024Hierarchical Open-Vocabulary 3D Scene Graphs for Language-Grounded Robot Navigationcited by 196
  9. 2022Hydra: A Real-time Spatial Perception System for 3D Scene Graph Construction and Optimizationcited by 178
  10. 2022Habitat-Matterport 3D Semantics Datasetcited by 110
  11. 2024SG-Nav: Online 3D Scene Graph Prompting for LLM-based Zero-shot Object Navigationcited by 109
  12. 2024Clio: Real-time Task-Driven Open-Set 3D Scene Graphscited by 81
20 more
  1. 2024Open3DSG: Open-Vocabulary 3D Scene Graphs from Point Clouds with Queryable Objects and Open-Set Relationshipscited by 80
  2. 2024ReMEmbR: Building and Reasoning Over Long-Horizon Spatio-Temporal Memory for Robot Navigationcited by 67
  3. 2025Open-Vocabulary Functional 3D Scene Graphs for Real-World Indoor Spacescited by 54
  4. 2024Dynamic Open-Vocabulary 3D Scene Graphs for Long-term Language-Guided Mobile Manipulationcited by 39
  5. 2024Khronos: A Unified Approach for Spatio-Temporal Metric-Semantic SLAM in Dynamic Environmentscited by 36
  6. 2025Mem2Ego: Empowering Vision-Language Models with Global-to-Ego Memory for Long-Horizon Embodied Navigationcited by 30
  7. 2024EmbodiedRAG: Dynamic 3D Scene Graph Retrieval for Efficient and Scalable Robot Task Planningcited by 24
  8. 2025Describe Anything Anywhere At Any Momentcited by 19
  9. 2024Open-Vocabulary Online Semantic Mapping for SLAMcited by 12
  10. 2025FSR-VLN: Fast and Slow Reasoning for Vision-Language Navigation with Hierarchical Multi-modal Scene Graphcited by 11
  11. 2025OpenIN: Open-Vocabulary Instance-Oriented Navigation in Dynamic Domestic Environmentscited by 10
  12. 2024Task-oriented Sequential Grounding and Navigation in 3D Scenescited by 9
  13. 2025ASHiTA: Automatic Scene-grounded HIerarchical Task Analysiscited by 8
  14. 2025DynamicGSG: Dynamic 3D Gaussian Scene Graphs for Environment Adaptationcited by 7
  15. 2025DualMap: Online Open-Vocabulary Semantic Mapping for Natural Language Navigation in Dynamic Changing Scenescited by 6
  16. 2025Vision to Geometry: 3D Spatial Memory for Sequential Embodied MLLM Reasoning and Explorationcited by 5
  17. 2026RAG-3DSG: Enhancing 3D Scene Graphs with Re-Shot Guided Retrieval-Augmented Generationcited by 5
  18. 2025Affordance RAG: Hierarchical Multimodal Retrieval with Affordance-Aware Embodied Memory for Mobile Manipulationcited by 3
  19. 2026INHerit-SG: Incremental Hierarchical Semantic Scene Graphs for Complex Embodied Retrievalcited by 2
  20. 2025ZING-3D: Zero-shot Incremental 3D Scene Graphs via Vision-Language Modelscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Move to Understand a 3D Scene: Bridging Visual Grounding and Exploration for Efficient and Versatile Embodied Navigation
  2. 2026AgentGrounder: Zero-Shot 3D Visual Pointcloud Grounding using Multimodal Language Models
  3. 2025GraphPad: Inference-Time 3D Scene Graph Updates for Embodied Question Answering
  4. 2025OpenGround: Planning-based Online Perception for Open-World 3D Visual Grounding
  5. 2026RenderMem: Rendering as Spatial Memory Retrieval
  6. 2026GSMem: 3D Gaussian Splatting as Persistent Spatial Memory for Zero-Shot Embodied Exploration and Reasoning
  7. 2026EPIC-Bench: A Perception-Centric Benchmark for Fine-Grained Embodied Visual Grounding in Vision-Language Models
  8. 2024Explore until Confident: Efficient Exploration for Embodied Question Answering