mlsanityData to 7 Oct 2026

Zero Experience Required: Plug & Play Modular Transfer Learning for Semantic Visual Navigation

arXiv 2202.02440 ↗First posted 5 Feb 2022cs.CV cs.AI cs.LGcited by 66

Not on the map. Why?

Abstract (arXiv)

In reinforcement learning for visual navigation, it is common to develop a model for each new task, and train that model from scratch with task-specific interactions in 3D environments. However, this process is expensive; massive amounts of interactions are needed for the model to generalize well. Moreover, this process is repeated whenever there is a change in the task type or the goal modality. We present a unified approach to visual navigation using a novel modular transfer learning model. Our model can effectively leverage its experience from one source task and apply it to multiple target tasks (e.g., ObjectNav, RoomNav, ViewNav) with various goal modalities (e.g., image, sketch, audio, label). Furthermore, our model enables zero-shot experience learning, whereby it can solve the target tasks without receiving any task-specific interactive training. Our experiments on multiple photorealistic datasets and challenging tasks show that our approach learns faster, generalizes better, and outperforms SoTA models by a significant margin.

Key facts. Cited by 66 papers in the map (data through 7 Oct 2026). Builds on 59 of its 87 references that are in the map. First posted on arXiv 5 Feb 2022.

Builds on

59 of 87 references.

References · 59

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2017Proximal Policy Optimization Algorithmscited by 14,441
  5. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  6. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  7. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
  8. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,282
  9. 2013Rich feature hierarchies for accurate object detection and semantic segmentationcited by 2,160
  10. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
  11. 2014On the Properties of Neural Machine Translation: Encoder-Decoder Approachescited by 1,303
  12. 2019Habitat: A Platform for Embodied AI Researchcited by 1,163
38 more
  1. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,013
  2. 2017AI2-THOR: An Interactive 3D Environment for Visual AIcited by 850
  3. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 734
  4. 2018Taskonomy: Disentangling Task Transfer Learningcited by 631
  5. 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 579
  6. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 573
  7. 2018On Evaluation of Embodied Navigation Agentscited by 526
  8. 2018Gibson Env: Real-World Perception for Embodied Agentscited by 477
  9. 2020Reinforcement Learning with Augmented Datacited by 452
  10. 2020Planning to Explore via Self-Supervised World Modelscited by 365
  11. 2019DD-PPO: Learning Near-Perfect PointGoal Navigators from 2.5 Billion Framescited by 352
  12. 2016Target-driven Visual Navigation in Indoor Scenes using Deep Reinforcement Learningcited by 351
  13. 20193D Scene Graph: A Structure for Unified Semantics, 3D Space, and Cameracited by 280
  14. 2017Zero-Shot Learning -- The Good, the Bad and the Uglycited by 254
  15. 2020ObjectNav Revisited: On Evaluation of Embodied Agents Navigating to Objectscited by 240
  16. 2018Semi-parametric Topological Memory for Navigationcited by 203
  17. 2020Neural Topological SLAM for Visual Navigationcited by 193
  18. 2022PONI: Potential Functions for ObjectGoal Navigation with Interaction-free Learningcited by 149
  19. 2018Visual Semantic Navigation using Scene Priorscited by 127
  20. 2020Occupancy Anticipation for Efficient Exploration and Navigationcited by 104
  21. 2018Learning to Navigate in Cities Without a Mapcited by 103
  22. 2019Look, Listen, and Act: Towards Audio-Visual Embodied Navigationcited by 95
  23. 2018Driving Policy Transfer via Modularity and Abstractioncited by 91
  24. 2018Learning to Learn How to Learn: Self-Adaptive Visual Navigation Using Meta-Learningcited by 91
  25. 2018Visual Representations for Semantic Target Driven Navigationcited by 89
  26. 2020Semantic Audio-Visual Navigationcited by 79
  27. 2021Auxiliary Tasks and Exploration Enable ObjectNavcited by 73
  28. 2017MINOS: Multimodal Indoor Simulator for Navigation in Complex Environmentscited by 66
  29. 2019Does computer vision matter for action?cited by 64
  30. 2021Memory-Augmented Reinforcement Learning for Image-Goal Navigationcited by 64
  31. 2021No RL, No Simulation: Learning to Navigate without Navigatingcited by 63
  32. 2021Learning to See before Learning to Act: Visual Pre-training for Manipulationcited by 50
  33. 2019Bayesian Relational Memory for Semantic Visual Navigationcited by 48
  34. 2020Environment-agnostic Multitask Learning for Natural Language Grounded Navigationcited by 43
  35. 2021Curious Representation Learning for Embodied Intelligencecited by 43
  36. 2019Unsupervised Reinforcement Learning of Transferable Meta-Skills for Embodied Navigationcited by 39
  37. 2020Seeing the Un-Scene: Learning Amodal Semantic Maps for Room Navigationcited by 34
  38. 2019SplitNet: Sim2Sim and Task2Task Transfer for Embodied Visual Navigationcited by 29

50 most cited of 59 All 59 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 9; 2023: 17; 2024: 20; 2025: 15; 2026: 5.

66 papers.

Sort:

By area:

  1. 2023Where are we in the search for an Artificial Visual Cortex for Embodied Intelligence?cited by 215
  2. 2022CoWs on Pasture: Baselines and Benchmarks for Language-Driven Zero-Shot Object Navigationcited by 194
  3. 2022ZSON: Zero-Shot Object-Goal Navigation using Multimodal Goal Embeddingscited by 177
  4. 2023Habitat Synthetic Scenes Dataset (HSSD-200): An Analysis of 3D Scene Scale and Realism Tradeoffs for ObjectGoal Navigationcited by 162
  5. 2023ESC: Exploration with Soft Commonsense Constraints for Zero-shot Object Navigationcited by 126
  6. 2024GOAT-Bench: A Benchmark for Multi-Modal Lifelong Navigationcited by 93
  7. 2022Offline Visual Representation Learning for Embodied Navigationcited by 74
  8. 2023OVRL-V2: A simple state-of-art baseline for ImageNav and ObjectNavcited by 68
  9. 2023A2Nav: Action-Aware Zero-Shot Robot Navigation by Exploiting Vision-and-Language Ability of Foundation Modelscited by 61
  10. 2024VoroNav: Voronoi-based Zero-shot Object Navigation with Large Language Modelcited by 55
  11. 2022Instance-Specific Image Goal Navigation: Training Embodied Agents to Find Object Instancescited by 53
  12. 2023Renderable Neural Radiance Map for Visual Navigationcited by 51
38 more
  1. 2023Navigating to Objects Specified by Imagescited by 50
  2. 2024GaussNav: Gaussian Splatting for Visual Navigationcited by 49
  3. 2022Retrospectives on the Embodied AI Workshopcited by 46
  4. 2025WMNav: Integrating Vision-Language Models into World Models for Object Goal Navigationcited by 42
  5. 2022Prompter: Utilizing Large Language Model Prompting for a Data Efficient Embodied Instruction Followingcited by 33
  6. 2024Prioritized Semantic Learning for Zero-shot Instance Navigationcited by 30
  7. 2022Last-Mile Embodied Visual Navigationcited by 25
  8. 2025Prior Does Matter: Visual Navigation via Denoising Diffusion Bridge Modelscited by 24
  9. 2023FGPrompt: Fine-grained Goal Prompting for Image-goal Navigationcited by 21
  10. 2023Omnidirectional Information Gathering for Knowledge Transfer-based Audio-Visual Navigationcited by 18
  11. 2024Zero-shot Object Navigation with Vision-Language Models Reasoningcited by 18
  12. 2023Object Goal Navigation with Recursive Implicit Mapscited by 16
  13. 2023End-to-End (Instance)-Image Goal Navigation through Correspondence as an Emergent Phenomenoncited by 16
  14. 2024SAME: Learning Generic Language-Guided Visual Navigation with State-Adaptive Mixture of Expertscited by 16
  15. 2024Instance-aware Exploration-Verification-Exploitation for Instance ImageGoal Navigationcited by 15
  16. 2024MemoNav: Working Memory Model for Visual Navigationcited by 13
  17. 2024Open-vocabulary Mobile Manipulation in Unseen Dynamic Environments with 3D Semantic Mapscited by 13
  18. 2024Tag Map: A Text-Based Map for Spatial Reasoning and Navigation with Large Language Modelscited by 12
  19. 2023What do we learn from a large-scale study of pre-trained visual representations in sim and real environments?cited by 11
  20. 2024Personalized Instance-based Navigation Toward User-Specific Objects in Realistic Environmentscited by 11
  21. 2025From Seeing to Experiencing: Scaling Navigation Foundation Models with Reinforcement Learningcited by 11
  22. 2025MMTL-UniAD: A Unified Framework for Multimodal and Multi-Task Learning in Assistive Driving Perceptioncited by 10
  23. 2025IGL-Nav: Incremental 3D Gaussian Localization for Image-goal Navigationcited by 10
  24. 2023Exploitation-Guided Exploration for Semantic Embodied Navigationcited by 9
  25. 2023Multiple Thinking Achieving Meta-Ability Decoupling for Object Navigationcited by 8
  26. 2023Co-NavGPT: Multi-Robot Cooperative Visual Semantic Navigation Using Vision Language Modelscited by 8
  27. 2024LOC-ZSON: Language-driven Object-Centric Zero-Shot Object Retrieval and Navigationcited by 8
  28. 2024FIction: 4D Future Interaction Prediction from Videocited by 8
  29. 2025DORAEMON: Decentralized Ontology-aware Reliable Agent with Enhanced Memory Oriented Navigationcited by 6
  30. 2022Intra-agent speech permits zero-shot task acquisitioncited by 5
  31. 2023Visual Hindsight Self-Imitation Learning for Interactive Navigationcited by 5
  32. 2024Efficient Policy Adaptation with Contrastive Prompt Ensemble for Embodied Agentscited by 5
  33. 2022Self-Supervised Object Goal Navigation with In-Situ Finetuningcited by 4
  34. 2023Leveraging Human Feedback to Evolve and Discover Novel Emergent Behaviors in Robot Swarmscited by 4
  35. 2024Feudal Networks for Visual Navigationcited by 4
  36. 2024NOLO: Navigate Only Look Oncecited by 3
  37. 2025Image-Goal Navigation Using Refined Feature Guidance and Scene Graph Enhancementcited by 3
  38. 2024Leveraging Large Language Model-based Room-Object Relationships Knowledge for Enhancing Multimodal-Input Object Goal Navigationcited by 2

50 most cited of 66 All 66 (JSON)