# Multi-Agent Embodied Visual Semantic Navigation with Scene Prior Knowledge

arXiv: https://arxiv.org/abs/2109.09531 · HTML: https://mlsanity.com/paper/2109.09531/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 38 references that are in the map.
- First posted on arXiv 20 Sep 2021.

## Builds on (most cited first, 30 of 30)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations](https://mlsanity.com/paper/1602.07332/) (2016)
- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environments](https://mlsanity.com/paper/1711.07280/) (2017)
- [AI2-THOR: An Interactive 3D Environment for Visual AI](https://mlsanity.com/paper/1712.05474/) (2017)
- [On Evaluation of Embodied Navigation Agents](https://mlsanity.com/paper/1807.06757/) (2018)
- [Object Goal Navigation using Goal-Oriented Semantic Exploration](https://mlsanity.com/paper/2007.00643/) (2020)
- [Embodied Question Answering](https://mlsanity.com/paper/1711.11543/) (2017)
- [Human-level performance in first-person multiplayer games with population-based deep reinforcement learning](https://mlsanity.com/paper/1807.01281/) (2018)
- [TarMAC: Targeted Multi-Agent Communication](https://mlsanity.com/paper/1810.11187/) (2018)
- [RoboTHOR: An Open Simulation-to-Real Embodied AI Platform](https://mlsanity.com/paper/2004.06799/) (2020)
- [Deep Decentralized Multi-task Multi-Agent Reinforcement Learning under Partial Observability](https://mlsanity.com/paper/1703.06182/) (2017)
- [Visual Semantic Navigation using Scene Priors](https://mlsanity.com/paper/1810.06543/) (2018)
- [Look, Listen, and Act: Towards Audio-Visual Embodied Navigation](https://mlsanity.com/paper/1912.11684/) (2019)
- [Learning to Learn How to Learn: Self-Adaptive Visual Navigation Using Meta-Learning](https://mlsanity.com/paper/1812.00971/) (2018)
- [Visual Representations for Semantic Target Driven Navigation](https://mlsanity.com/paper/1805.06066/) (2018)
- [A Cordial Sync: Going Beyond Marginal Policies for Multi-Agent Embodied Tasks](https://mlsanity.com/paper/2007.04979/) (2020)
- [Two Body Problem: Collaborative Visual Task Completion](https://mlsanity.com/paper/1904.05879/) (2019)
- [SSCNav: Confidence-Aware Semantic Scene Completion for Visual Semantic Navigation](https://mlsanity.com/paper/2012.04512/) (2020)
- [Visual Navigation with Spatial Attention](https://mlsanity.com/paper/2104.09807/) (2021)
- [Unsupervised Reinforcement Learning of Transferable Meta-Skills for Embodied Navigation](https://mlsanity.com/paper/1911.07450/) (2019)
- [Exploiting Scene-specific Features for Object Goal Navigation](https://mlsanity.com/paper/2008.09403/) (2020)
- [Towards Target-Driven Visual Navigation in Indoor Scenes via Generative Imitation Learning](https://mlsanity.com/paper/2009.14509/) (2020)
- [Visual Hide and Seek](https://mlsanity.com/paper/1910.07882/) (2019)
- [Efficient Robotic Object Search via HIEM: Hierarchical Policy Learning with Intrinsic-Extrinsic Modeling](https://mlsanity.com/paper/2010.08596/) (2020)
- [Reinforcement Learning-based Visual Navigation with Information-Theoretic Regularization](https://mlsanity.com/paper/1912.04078/) (2019)
- [Learning Generalizable Visual Representations via Interactive Gameplay](https://mlsanity.com/paper/1912.08195/) (2019)
- [Learning for Visual Navigation by Imagining the Success](https://mlsanity.com/paper/2103.00446/) (2021)

## Built on by (most cited first, 10 of 10)

- [Search for or Navigate to? Dual Adaptive Thinking for Object Navigation](https://mlsanity.com/paper/2208.00553/) (2022)
- [Learning Efficient Multi-Agent Cooperative Visual Exploration](https://mlsanity.com/paper/2110.05734/) (2021)
- [VIKI-R: Coordinating Embodied Multi-Agent Cooperation via Reinforcement Learning](https://mlsanity.com/paper/2506.09049/) (2025)
- [Dual Latent Memory for Visual Multi-agent System](https://mlsanity.com/paper/2602.00471/) (2026)
- [Co-NavGPT: Multi-Robot Cooperative Visual Semantic Navigation Using Vision Language Models](https://mlsanity.com/paper/2310.07937/) (2023)
- [LEMMA: Learning Language-Conditioned Multi-Robot Manipulation](https://mlsanity.com/paper/2308.00937/) (2023)
- [Heterogeneous Embodied Multi-Agent Collaboration](https://mlsanity.com/paper/2307.13957/) (2023)
- [Visual Semantic Navigation with Real Robots](https://mlsanity.com/paper/2311.16623/) (2023)
- [Does Peer Observation Help? Vision-Sharing Collaboration for Vision-Language Navigation](https://mlsanity.com/paper/2603.20804/) (2026)
- [MASP: Scalable GNN-based Planning for Multi-Agent Navigation](https://mlsanity.com/paper/2312.02522/) (2023)

## Abstract (arXiv)

In visual semantic navigation, the robot navigates to a target object with egocentric visual observations and the class label of the target is given. It is a meaningful task inspiring a surge of relevant research. However, most of the existing models are only effective for single-agent navigation, and a single agent has low efficiency and poor fault tolerance when completing more complicated tasks. Multi-agent collaboration can improve the efficiency and has strong application potentials. In this paper, we propose the multi-agent visual semantic navigation, in which multiple agents collaborate with others to find multiple target objects. It is a challenging task that requires agents to learn reasonable collaboration strategies to perform efficient exploration under the restrictions of communication bandwidth. We develop a hierarchical decision framework based on semantic mapping, scene prior knowledge, and communication mechanism to solve this task. The results of testing experiments in unseen scenes with both known objects and unknown objects illustrate the higher accuracy and efficiency of the proposed model compared with the single-agent model.