# Bayesian Relational Memory for Semantic Visual Navigation

arXiv: https://arxiv.org/abs/1909.04306 · HTML: https://mlsanity.com/paper/1909.04306/

## Key facts

- Cited by 48 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Sep 2019.

## Built on by (most cited first, 30 of 48)

- [Structured Scene Memory for Vision-Language Navigation](https://mlsanity.com/paper/2103.03454/) (2021)
- [Beyond A*: Better Planning with Transformers via Search Dynamics Bootstrapping](https://mlsanity.com/paper/2402.14083/) (2024)
- [Zero Experience Required: Plug & Play Modular Transfer Learning for Semantic Visual Navigation](https://mlsanity.com/paper/2202.02440/) (2022)
- [Memory-Augmented Reinforcement Learning for Image-Goal Navigation](https://mlsanity.com/paper/2101.05181/) (2021)
- [No RL, No Simulation: Learning to Navigate without Navigating](https://mlsanity.com/paper/2110.09470/) (2021)
- [Hierarchical Representations and Explicit Memory: Learning Effective Navigation Policies on 3D Scene Graphs using Graph Neural Networks](https://mlsanity.com/paper/2108.01176/) (2021)
- [VTNet: Visual Transformer Network for Object Goal Navigation](https://mlsanity.com/paper/2105.09447/) (2021)
- [Navigating to Objects Specified by Images](https://mlsanity.com/paper/2304.01192/) (2023)
- [Visual Navigation with Spatial Attention](https://mlsanity.com/paper/2104.09807/) (2021)
- [Dualformer: Controllable Fast and Slow Thinking by Learning with Randomized Reasoning Traces](https://mlsanity.com/paper/2410.09918/) (2024)
- [Continuous Scene Representations for Embodied AI](https://mlsanity.com/paper/2203.17251/) (2022)
- [Move2Hear: Active Audio-Visual Source Separation](https://mlsanity.com/paper/2105.07142/) (2021)
- [Vision-Language Navigation: A Survey and Taxonomy](https://mlsanity.com/paper/2108.11544/) (2021)
- [Hierarchical Object-to-Zone Graph for Object Navigation](https://mlsanity.com/paper/2109.02066/) (2021)
- [SGUIE-Net: Semantic Attention Guided Underwater Image Enhancement with Multi-Scale Perception](https://mlsanity.com/paper/2201.02832/) (2022)
- [Episodic Memory Question Answering](https://mlsanity.com/paper/2205.01652/) (2022)
- [Towards Versatile Embodied Navigation](https://mlsanity.com/paper/2210.16822/) (2022)
- [Reasoning with Scene Graphs for Robot Planning under Partial Observability](https://mlsanity.com/paper/2202.10432/) (2022)
- [Learning Efficient Multi-Agent Cooperative Visual Exploration](https://mlsanity.com/paper/2110.05734/) (2021)
- [Selective Visual Representations Improve Convergence and Generalization for Embodied AI](https://mlsanity.com/paper/2311.04193/) (2023)
- [Modeling Dynamic Environments with Scene Graph Memory](https://mlsanity.com/paper/2305.17537/) (2023)
- [Navigating to Objects in Unseen Environments by Distance Prediction](https://mlsanity.com/paper/2202.03735/) (2022)
- [Environment Predictive Coding for Embodied Agents](https://mlsanity.com/paper/2102.02337/) (2021)
- [Memory-Centric Embodied Question Answering](https://mlsanity.com/paper/2505.13948/) (2025)
- [Agent-Centric Relation Graph for Object Visual Navigation](https://mlsanity.com/paper/2111.14422/) (2021)
- [Aligning Knowledge Graph with Visual Perception for Object-goal Navigation](https://mlsanity.com/paper/2402.18892/) (2024)
- [Multiple Thinking Achieving Meta-Ability Decoupling for Object Navigation](https://mlsanity.com/paper/2302.01520/) (2023)
- [Hierarchical and Partially Observable Goal-driven Policy Learning with Goals Relational Graph](https://mlsanity.com/paper/2103.01350/) (2021)
- [Lifelong Topological Visual Navigation](https://mlsanity.com/paper/2110.08488/) (2021)
- [A Simple Approach for Visual Rearrangement: 3D Mapping and Semantic Search](https://mlsanity.com/paper/2206.13396/) (2022)

## Abstract (arXiv)

We introduce a new memory architecture, Bayesian Relational Memory (BRM), to improve the generalization ability for semantic visual navigation agents in unseen environments, where an agent is given a semantic target to navigate towards. BRM takes the form of a probabilistic relation graph over semantic entities (e.g., room types), which allows (1) capturing the layout prior from training environments, i.e., prior knowledge, (2) estimating posterior layout at test time, i.e., memory update, and (3) efficient planning for navigation, altogether. We develop a BRM agent consisting of a BRM module for producing sub-goals and a goal-conditioned locomotion module for control. When testing in unseen environments, the BRM agent outperforms baselines that do not explicitly utilize the probabilistic relational memory structure