mlsanityData to 7 Oct 2026

Balancing Performance and Efficiency in Zero-shot Robotic Navigation

arXiv 2406.03015 ↗First posted 5 Jun 2024cs.RO cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

We present an optimization study of the Vision-Language Frontier Maps (VLFM) applied to the Object Goal Navigation task in robotics. Our work evaluates the efficiency and performance of various vision-language models, object detectors, segmentation models, and multi-modal comprehension and Visual Question Answering modules. Using the textitval-mini and textitval splits of Habitat-Matterport 3D dataset, we conduct experiments on a desktop with limited VRAM. We propose a solution that achieves a higher success rate (+1.55%) improving over the VLFM BLIP-2 baseline without substantial success-weighted path length loss while requiring 2.3 times less video memory. Our findings provide insights into balancing model performance and computational efficiency, suggesting effective deployment strategies for resource-limited environments.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 31 of its 33 references that are in the map. First posted on arXiv 5 Jun 2024.

Builds on

31 of 33 references.

References · 31

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2023Visual Instruction Tuningcited by 7,792
  5. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  6. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  7. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  8. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  9. 2022A ConvNet for the 2020scited by 3,545
  10. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
  11. 2023Qwen Technical Reportcited by 3,273
  12. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
19 more
  1. 2015VQA: Visual Question Answeringcited by 2,489
  2. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
  3. 2022YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectorscited by 978
  4. 2022PaLI: A Jointly-Scaled Multilingual Language-Image Modelcited by 762
  5. 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 579
  6. 2018On Evaluation of Embodied Navigation Agentscited by 526
  7. 2022Image as a Foreign Language: BEiT Pretraining for All Vision and Vision-Language Taskscited by 510
  8. 2018Gibson Env: Real-World Perception for Embodied Agentscited by 477
  9. 2023Faster Segment Anything: Towards Lightweight SAM for Mobile Applicationscited by 345
  10. 2024MA-LMM: Memory-Augmented Large Multimodal Model for Long-Term Video Understandingcited by 223
  11. 2023VLFM: Vision-Language Frontier Maps for Zero-Shot Semantic Navigationcited by 201
  12. 2022CoWs on Pasture: Baselines and Benchmarks for Language-Driven Zero-Shot Object Navigationcited by 194
  13. 2022ZSON: Zero-Shot Object-Goal Navigation using Multimodal Goal Embeddingscited by 177
  14. 2023ESC: Exploration with Soft Commonsense Constraints for Zero-shot Object Navigationcited by 126
  15. 2023PIRLNav: Pretraining with Imitation and RL Finetuning for ObjectNavcited by 91
  16. 2023Can an Embodied Agent Find Your "Cat-shaped Mug"? LLM-Guided Exploration for Zero-Shot Object Navigationcited by 74
  17. 2023How To Not Train Your Dragon: Training-free Embodied Object Goal Navigation with Semantic Frontierscited by 53
  18. 2022VER: Scaling On-Policy RL Leads to the Emergence of Navigation in Embodied Rearrangementcited by 18
  19. 2023Frontier Semantic Exploration for Visual Target Navigationcited by 9

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025DyNaVLM: Zero-Shot Vision-Language Navigation System with Dynamic Viewpoints and Self-Refining Graph Memory
  2. 2026Exploring Bottlenecks in VLM-LLM Navigation: How 3D Scene Understanding Capability Impacts Zero-Shot VLN
  3. 2024VLN-Game: Vision-Language Equilibrium Search for Zero-Shot Semantic Navigation
  4. 2026ViTL: Temporal Logic-Guided Zero-Shot Natural Language Navigation via Vision-Language Models
  5. 2026OpenFrontier: General Navigation with Visual-Language Grounded Frontiers
  6. 2025FOM-Nav: Frontier-Object Maps for Object Goal Navigation
  7. 2026C2Nav: Compare Before You Commit for Zero-Shot Vision-and-Language Navigation
  8. 2026LightZeroNav: Zero-Shot Vision Language Navigation in Continuous Environments Based on Lightweight VLMs