# Learning Semantic-Agnostic and Spatial-Aware Representation for Generalizable Visual-Audio Navigation

arXiv: https://arxiv.org/abs/2304.10773 · HTML: https://mlsanity.com/paper/2304.10773/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Apr 2023.

## Built on by (most cited first, 9 of 9)

- [Find What You Want: Learning Demand-conditioned Object Attribute Space for Demand-driven Navigation](https://mlsanity.com/paper/2309.08138/) (2023)
- [Audio-Guided Dynamic Modality Fusion with Stereo-Aware Attention for Audio-Visual Navigation](https://mlsanity.com/paper/2509.16924/) (2025)
- [Embodied Navigation with Auxiliary Task of Action Description Prediction](https://mlsanity.com/paper/2510.21809/) (2025)
- [Visual Semantic Navigation with Real Robots](https://mlsanity.com/paper/2311.16623/) (2023)
- [Audio-Guided Visual Perception for Audio-Visual Navigation](https://mlsanity.com/paper/2510.11760/) (2025)
- [MAGIC: Map-Guided Few-Shot Audio-Visual Acoustics Modeling](https://mlsanity.com/paper/2405.13860/) (2024)
- [Generalizable Audio-Visual Navigation via Binaural Difference Attention and Action Transition Prediction](https://mlsanity.com/paper/2604.05007/) (2026)
- [RescueBench: Can Embodied Agents Save Lives in the Wild ?](https://mlsanity.com/paper/2606.01848/) (2026)
- [Two-Stage Multi-modal Learning for Dynamic Audio-Visual Navigation: From Physical Alignment to Scene Aggregation](https://mlsanity.com/paper/2610.14077/) (2026)

## Abstract (arXiv)

Visual-audio navigation (VAN) is attracting more and more attention from the robotic community due to its broad applications, e.g., household robots and rescue robots. In this task, an embodied agent must search for and navigate to the sound source with egocentric visual and audio observations. However, the existing methods are limited in two aspects: 1) poor generalization to unheard sound categories; 2) sample inefficient in training. Focusing on these two problems, we propose a brain-inspired plug-and-play method to learn a semantic-agnostic and spatial-aware representation for generalizable visual-audio navigation. We meticulously design two auxiliary tasks for respectively accelerating learning representations with the above-desired characteristics. With these two auxiliary tasks, the agent learns a spatially-correlated representation of visual and audio inputs that can be applied to work on environments with novel sounds and maps. Experiment results on realistic 3D scenes (Replica and Matterport3D) demonstrate that our method achieves better generalization performance when zero-shot transferred to scenes with unseen maps and unheard sound categories.