mlsanityData to 13 Oct 2026

Find Everything: A General Vision Language Model Approach to Multi-Object Search

arXiv 2410.00388 ↗First posted 1 Oct 2024cs.ROcited by 4

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

The Multi-Object Search (MOS) problem involves navigating to a sequence of locations to maximize the likelihood of finding target objects while minimizing travel costs. In this paper, we introduce a novel approach to the MOS problem, called Finder, which leverages vision language models (VLMs) to locate multiple objects across diverse environments. Specifically, our approach introduces multi-channel score maps to track and reason about multiple objects simultaneously during navigation, along with a score map technique that combines scene-level and object-level semantic correlations. Experiments in both simulated and real-world settings showed that Finder outperforms existing methods using deep reinforcement learning and VLMs. Ablation and scalability studies further validated our design choices and robustness with increasing numbers of target objects, respectively. Website: https://find-all-my-things.github.io/

Key facts. Cited by 4 papers in the map (data through 13 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 40 of its 58 references that are in the map. First posted on arXiv 1 Oct 2024.

Builds on

40 of 58 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024OpenFMNav: Towards Open-Set Zero-Shot Object Navigation via Vision-Language Foundation Models16 Feb
  2. 2023VLFM: Vision-Language Frontier Maps for Zero-Shot Semantic Navigation6 Dec
  3. 2023Multimodal Large Language Model for Visual Navigation12 Oct

Other references · 37

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,550
  2. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,103
  3. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,299
  4. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,352
  5. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,360
  6. 2019Habitat: A Platform for Embodied AI Researchcited by 1,169
  7. 2022YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectorscited by 979
  8. 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 583
  9. 2018On Evaluation of Embodied Navigation Agentscited by 531
  10. 2020Object Goal Navigation using Goal-Oriented Semantic Explorationcited by 421
  11. 2022CoWs on Pasture: Baselines and Benchmarks for Language-Driven Zero-Shot Object Navigationcited by 194
  12. 2022ZSON: Zero-Shot Object-Goal Navigation using Multimodal Goal Embeddingscited by 177
25 more
  1. 2024InstructNav: Zero-shot System for Generic Instruction Navigation in Unexplored Environmentcited by 152
  2. 2023ESC: Exploration with Soft Commonsense Constraints for Zero-shot Object Navigationcited by 126
  3. 2023L3MVN: Leveraging Large Language Models for Visual Target Navigationcited by 118
  4. 2020MultiON: Benchmarking Semantic Map Memory using Multi-Object Navigationcited by 95
  5. 2023Navigation with Large Language Models: Semantic Guesswork as a Heuristic for Planningcited by 88
  6. 2023Bridging Zero-shot Object Navigation and Foundation Models through Pixel-Guided Navigation Skillcited by 75
  7. 2023Can an Embodied Agent Find Your "Cat-shaped Mug"? LLM-Guided Exploration for Zero-Shot Object Navigationcited by 74
  8. 2024VoroNav: Voronoi-based Zero-shot Object Navigation with Large Language Modelcited by 55
  9. 2023MobileSAMv2: Faster Segment Anything to Everythingcited by 42
  10. 2022Learning Active Camera for Multi-Object Navigationcited by 20
  11. 2024NavFormer: A Transformer Architecture for Robot Target-Driven Navigation in Unknown and Dynamic Environmentscited by 20
  12. 2022VER: Scaling On-Policy RL Leads to the Emergence of Navigation in Embodied Rearrangementcited by 18
  13. 2024SEEK: Semantic Reasoning for Object Goal Navigation in Real World Inspection Taskscited by 18
  14. 2021Teaching Agents how to Map: Spatial Reasoning for Multi-Object Navigationcited by 17
  15. 2022Learning Long-Horizon Robot Exploration Strategies for Multi-Object Search in Continuous Action Spacescited by 14
  16. 2023Learning Hierarchical Interactive Multi-Object Search for Mobile Manipulationcited by 12
  17. 2021Deep Reinforcement Learning for Decentralized Multi-Robot Exploration With Macro Actionscited by 10
  18. 2023A System for Generalized 3D Multi-Object Searchcited by 9
  19. 2022Robots Autonomously Detecting People: A Multimodal Deep Contrastive Learning Method Robust to Intraclass Variationscited by 6
  20. 2020Robots Understanding Contextual Information in Human-Centered Environments using Weakly Supervised Mask Data Distillationcited by 5
  21. 2024Multi-Object Navigation in real environments using hybrid policiescited by 3
  22. 2024Language-Based Augmentation to Address Shortcut Learning in Object Goal Navigationcited by 3
  23. 2024LDTrack: Dynamic People Tracking by Service Robots using Diffusion Modelscited by 3
  24. 2021A Scavenger Hunt for Service Robotscited by 1
  25. 2024Probabilistically Informed Robot Object Search with Multiple Regionscited by 1

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2025: 3; 2026: 1.

4 papers.

  1. 2025MoManipVLA: Transferring Vision-language-action Models for General Mobile Manipulationcited by 34
  2. 2025Embodied Crowd Countingcited by 1
  3. 2025DIV-Nav: Open-Vocabulary Spatial Relationships for Multi-Object Navigationcited by 1
  4. 2026VoLN: Vision-Only Long-Horizon Navigation---Paradigm, Benchmark, and Methodcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2025STRIVE: Structured Representation Integrating VLM Reasoning for Efficient Object Navigation
  2. 2025COMRES-VLM: Coordinated Multi-Robot Exploration and Search using Vision Language Models
  3. 2025FlySearch: Exploring how vision-language models explore
  4. 2025osmAG-LLM: Zero-Shot Open-Vocabulary Object Navigation via Semantic Maps and Large Language Models Reasoning
  5. 2026GoalVLM: VLM-driven Object Goal Navigation for Multi-Agent System
  6. 2024VLN-Game: Vision-Language Equilibrium Search for Zero-Shot Semantic Navigation
  7. 2025GET: Goal-directed Exploration and Targeting for Large-Scale Unknown Environments
  8. 2026OpenFrontier: General Navigation with Visual-Language Grounded Frontiers