mlsanityData to 13 Oct 2026

Collaborative Visual Navigation

arXiv 2107.01151 ↗First posted 2 Jul 2021cs.CV cs.AI cs.ROcited by 18

Not on the map. Why?

Abstract (arXiv)

As a fundamental problem for Artificial Intelligence, multi-agent system (MAS) is making rapid progress, mainly driven by multi-agent reinforcement learning (MARL) techniques. However, previous MARL methods largely focused on grid-world like or game environments; MAS in visually rich environments has remained less explored. To narrow this gap and emphasize the crucial role of perception in MAS, we propose a large-scale 3D dataset, CollaVN, for multi-agent visual navigation (MAVN). In CollaVN, multiple agents are entailed to cooperatively navigate across photo-realistic environments to reach target locations. Diverse MAVN variants are explored to make our problem more general. Moreover, a memory-augmented communication framework is proposed. Each agent is equipped with a private, external memory to persistently store communication information. This allows agents to make better use of their past communication information, enabling more efficient collaboration and robust long-term planning. In our experiments, several baselines and evaluation metrics are designed. We also empirically verify the efficacy of our proposed MARL approach across different MAVN task settings.

Key facts. Cited by 18 papers in the map (data through 13 Oct 2026). Builds on 71 of its 97 references that are in the map. First posted on arXiv 2 Jul 2021.

Builds on

71 of 97 references.

References · 71

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,950
  2. 2017Proximal Policy Optimization Algorithmscited by 14,592
  3. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,352
  4. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,223
  5. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,940
  6. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,525
  7. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,360
  8. 2019Habitat: A Platform for Embodied AI Researchcited by 1,169
  9. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,093
  10. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,016
  11. 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 910
  12. 2017AI2-THOR: An Interactive 3D Environment for Visual AIcited by 859
38 more
  1. 2017Counterfactual Multi-Agent Policy Gradientscited by 739
  2. 2018QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 700
  3. 2017Sim-to-Real Transfer of Robotic Control with Dynamics Randomizationcited by 622
  4. 2019The StarCraft Multi-Agent Challengecited by 592
  5. 2016Learning to Communicate with Deep Multi-Agent Reinforcement Learningcited by 566
  6. 2019Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithmscited by 555
  7. 2018On Evaluation of Embodied Navigation Agentscited by 531
  8. 2018Gibson Env: Real-World Perception for Embodied Agentscited by 477
  9. 2016Semantic Scene Completion from a Single Depth Imagecited by 469
  10. 2020Object Goal Navigation using Goal-Oriented Semantic Explorationcited by 421
  11. 2016Learning Multiagent Communication with Backpropagationcited by 359
  12. 2016Target-driven Visual Navigation in Indoor Scenes using Deep Reinforcement Learningcited by 353
  13. 2017Embodied Question Answeringcited by 335
  14. 2020Learning to Explore using Active Neural SLAMcited by 308
  15. 2020Is Independent Learning All You Need in the StarCraft Multi-Agent Challenge?cited by 286
  16. 2018Human-level performance in first-person multiplayer games with population-based deep reinforcement learningcited by 260
  17. 2018Actor-Attention-Critic for Multi-Agent Reinforcement Learningcited by 260
  18. 2017Emergence of Grounded Compositional Language in Multi-Agent Populationscited by 259
  19. 2018Touchdown: Natural Language Navigation and Spatial Reasoning in Visual Street Environmentscited by 238
  20. 2017Cognitive Mapping and Planning for Visual Navigationcited by 209
  21. 2018TarMAC: Targeted Multi-Agent Communicationcited by 204
  22. 2020Neural Topological SLAM for Visual Navigationcited by 193
  23. 2018Fully Decentralized Multi-Agent Reinforcement Learning with Networked Agentscited by 190
  24. 2020RoboTHOR: An Open Simulation-to-Real Embodied AI Platformcited by 190
  25. 2016Multi-Agent Cooperation and the Emergence of (Natural) Languagecited by 181
  26. 2018A Survey and Critique of Multiagent Deep Reinforcement Learningcited by 174
  27. 2018Learning Attentional Communication for Multi-Agent Cooperationcited by 172
  28. 2020When2com: Multi-Agent Perception via Communication Graph Groupingcited by 157
  29. 2020iGibson 1.0: a Simulation Environment for Interactive Tasks in Large Realistic Scenescited by 154
  30. 2018Graph Convolutional Reinforcement Learningcited by 150
  31. 2020Who2com: Collaborative Perception via Learnable Handshake Communicationcited by 127
  32. 2017Autonomous Agents Modelling Other Agents: A Comprehensive Survey and Open Problemscited by 126
  33. 2017Emergent Complexity via Multi-Agent Competitioncited by 118
  34. 2018Learning when to Communicate at Scale in Multiagent Cooperative and Competitive Taskscited by 117
  35. 2017Stabilising Experience Replay for Deep Multi-Agent Reinforcement Learningcited by 114
  36. 2018Building Generalizable Agents with a Realistic and Rich 3D Environmentcited by 109
  37. 2021Structured Scene Memory for Vision-Language Navigationcited by 95
  38. 2017Multiagent Bidirectionally-Coordinated Nets: Emergence of Human-level Coordination in Learning to Play StarCraft Combat Gamescited by 93

50 most cited of 71 All 71 (JSON)

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2021: 1; 2022: 6; 2023: 6; 2024: 2; 2025: 0; 2026: 3.

18 papers.

Sort:

By area:

  1. 2023DSVT: Dynamic Sparse Voxel Transformer with Rotated Setscited by 122
  2. 2022Target-Driven Structured Transformer Planner for Vision-Language Navigationcited by 58
  3. 2022CAGroup3D: Class-Aware Grouping for 3D Object Detection on Point Cloudscited by 58
  4. 2022Counterfactual Cycle-Consistent Learning for Instruction Following and Generation in Vision-Language Navigationcited by 50
  5. 2022RBGNet: Ray-based Grouping for 3D Object Detectioncited by 40
  6. 2022Towards Versatile Embodied Navigationcited by 26
  7. 2021Learning Efficient Multi-Agent Cooperative Visual Explorationcited by 19
  8. 2023Asynchronous Multi-Agent Reinforcement Learning for Efficient Real-Time Multi-Robot Cooperative Explorationcited by 17
  9. 2023LEMMA: Learning Language-Conditioned Multi-Robot Manipulationcited by 7
  10. 2023Heterogeneous Embodied Multi-Agent Collaborationcited by 6
  11. 2023Visual Hindsight Self-Imitation Learning for Interactive Navigationcited by 5
  12. 2024Weak-to-Strong 3D Object Detection with X-Ray Distillationcited by 5
6 more
  1. 2024MAexp: A Generic Platform for RL-based Multi-Agent Explorationcited by 5
  2. 2022CH-MARL: A Multimodal Benchmark for Cooperative, Heterogeneous Multi-Agent Reinforcement Learningcited by 4
  3. 2026Does Peer Observation Help? Vision-Sharing Collaboration for Vision-Language Navigationcited by 2
  4. 2023Cooperative Multi-Agent Learning for Navigation via Structured State Abstractioncited by 1
  5. 2026Cross-Modal Navigation with Multi-Agent Reinforcement Learningcited by 0
  6. 2026Systematic Multi-Agent Vision-and-Language Navigation: Formulation, Benchmark, and Methodcited by 0