Bayesian Relational Memory for Semantic Visual Navigation
Not on the map. Why?
Abstract (arXiv)
We introduce a new memory architecture, Bayesian Relational Memory (BRM), to improve the generalization ability for semantic visual navigation agents in unseen environments, where an agent is given a semantic target to navigate towards. BRM takes the form of a probabilistic relation graph over semantic entities (e.g., room types), which allows (1) capturing the layout prior from training environments, i.e., prior knowledge, (2) estimating posterior layout at test time, i.e., memory update, and (3) efficient planning for navigation, altogether. We develop a BRM agent consisting of a BRM module for producing sub-goals and a goal-conditioned locomotion module for control. When testing in unseen environments, the BRM agent outperforms baselines that do not explicitly utilize the probabilistic relational memory structure
Key facts. Cited by 48 papers in the map (data through 7 Oct 2026). First posted on arXiv 10 Sep 2019.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 17; 2022: 12; 2023: 7; 2024: 4; 2025: 3; 2026: 5.
48 papers.
Sort:
By area:
- 2021Structured Scene Memory for Vision-Language Navigationcited by 95
- 2024Beyond A*: Better Planning with Transformers via Search Dynamics Bootstrappingcited by 89
- 2022Zero Experience Required: Plug & Play Modular Transfer Learning for Semantic Visual Navigationcited by 66
- 2021Memory-Augmented Reinforcement Learning for Image-Goal Navigationcited by 64
- 2021No RL, No Simulation: Learning to Navigate without Navigatingcited by 63
- 2021VTNet: Visual Transformer Network for Object Goal Navigationcited by 50
- 2021Hierarchical Representations and Explicit Memory: Learning Effective Navigation Policies on 3D Scene Graphs using Graph Neural Networkscited by 50
- 2023Navigating to Objects Specified by Imagescited by 50
- 2021Visual Navigation with Spatial Attentioncited by 43
- 2024Dualformer: Controllable Fast and Slow Thinking by Learning with Randomized Reasoning Tracescited by 40
- 2022Continuous Scene Representations for Embodied AIcited by 39
- 2021Move2Hear: Active Audio-Visual Source Separationcited by 35
36 more
- 2021Hierarchical Object-to-Zone Graph for Object Navigationcited by 34
- 2021Vision-Language Navigation: A Survey and Taxonomycited by 33
- 2022SGUIE-Net: Semantic Attention Guided Underwater Image Enhancement with Multi-Scale Perceptioncited by 32
- 2022Episodic Memory Question Answeringcited by 32
- 2022Towards Versatile Embodied Navigationcited by 26
- 2022Reasoning with Scene Graphs for Robot Planning under Partial Observabilitycited by 25
- 2021Learning Efficient Multi-Agent Cooperative Visual Explorationcited by 19
- 2023Selective Visual Representations Improve Convergence and Generalization for Embodied AIcited by 19
- 2023Modeling Dynamic Environments with Scene Graph Memorycited by 18
- 2022Navigating to Objects in Unseen Environments by Distance Predictioncited by 15
- 2021Environment Predictive Coding for Embodied Agentscited by 12
- 2025Memory-Centric Embodied Question Answeringcited by 12
- 2021Agent-Centric Relation Graph for Object Visual Navigationcited by 10
- 2023Multiple Thinking Achieving Meta-Ability Decoupling for Object Navigationcited by 8
- 2024Aligning Knowledge Graph with Visual Perception for Object-goal Navigationcited by 8
- 2021Hierarchical and Partially Observable Goal-driven Policy Learning with Goals Relational Graphcited by 7
- 2021Lifelong Topological Visual Navigationcited by 7
- 2022A Simple Approach for Visual Rearrangement: 3D Mapping and Semantic Searchcited by 6
- 2025ObjectReact: Learning Object-Relative Control for Visual Navigationcited by 5
- 2022Spatial Relation Graph and Graph Convolutional Network for Object Goal Navigationcited by 3
- 2022A General Purpose Supervisory Signal for Embodied Agentscited by 3
- 2023ReVoLT: Relational Reasoning and Voronoi Local Graph Planning for Target-driven Navigationcited by 3
- 2023Implicit Obstacle Map-driven Indoor Navigation Model for Robust Obstacle Avoidancecited by 3
- 2021Deep Learning for Embodied Vision Navigation: A Surveycited by 2
- 2021Graph Convolutional Memory using Topological Priorscited by 1
- 2022Movement Analytics: Current Status, Application to Manufacturing, and Future Prospects from an AI Perspectivecited by 1
- 2026Memory-Guided View Refinement for Dynamic Human-in-the-loop EQAcited by 1
- 2026Semantic Audio-Visual Navigation in Continuous Environmentscited by 1
- 2021Symmetry-aware Neural Architecture for Embodied Visual Navigationcited by 0
- 2022Predicting Topological Maps for Visual Navigation in Unexplored Environmentscited by 0
- 2023Pseudo-Trilateral Adversarial Training for Domain Adaptive Traversability Predictioncited by 0
- 2024Semantic Environment Atlas for Object-Goal Navigationcited by 0
- 2025UAS Visual Navigation in Large and Unseen Environments via a Meta Agentcited by 0
- 2026Robo-Cortex: A Self-Evolving Embodied Agent via Dual-Grain Cognitive Memory and Autonomous Knowledge Inductioncited by 0
- 2026PixelLoop: Shortcut Topological Navigation with Pixel-Level Loopscited by 0
- 2026RAO-Nav: Probing Omni-Language Models for Zero-shot Semantic Audio-Visual Navigationcited by 0
- 2026RAO-Nav: Probing Omni-Language Models for Zero-shot Semantic Audio-Visual Navigation26 Sep 2026
- 2026PixelLoop: Shortcut Topological Navigation with Pixel-Level Loops14 Jul 2026
- 2026Robo-Cortex: A Self-Evolving Embodied Agent via Dual-Grain Cognitive Memory and Autonomous Knowledge Induction18 May 2026
- 2026Semantic Audio-Visual Navigation in Continuous Environments20 Mar 2026
- 2026Memory-Guided View Refinement for Dynamic Human-in-the-loop EQA10 Mar 2026
- 2025ObjectReact: Learning Object-Relative Control for Visual Navigation11 Sep 2025
- 2025Memory-Centric Embodied Question Answering20 May 2025
- 2025UAS Visual Navigation in Large and Unseen Environments via a Meta Agent20 Mar 2025
- 2024Dualformer: Controllable Fast and Slow Thinking by Learning with Randomized Reasoning Traces13 Oct 2024
- 2024Semantic Environment Atlas for Object-Goal Navigation5 Oct 2024
- 2024Aligning Knowledge Graph with Visual Perception for Object-goal Navigation29 Feb 2024
- 2024Beyond A*: Better Planning with Transformers via Search Dynamics Bootstrapping21 Feb 2024
36 more
- 2023Selective Visual Representations Improve Convergence and Generalization for Embodied AI7 Nov 2023
- 2023Implicit Obstacle Map-driven Indoor Navigation Model for Robust Obstacle Avoidance24 Aug 2023
- 2023Pseudo-Trilateral Adversarial Training for Domain Adaptive Traversability Prediction26 Jun 2023
- 2023Modeling Dynamic Environments with Scene Graph Memory27 May 2023
- 2023Navigating to Objects Specified by Images3 Apr 2023
- 2023Multiple Thinking Achieving Meta-Ability Decoupling for Object Navigation3 Feb 2023
- 2023ReVoLT: Relational Reasoning and Voronoi Local Graph Planning for Target-driven Navigation6 Jan 2023
- 2022A General Purpose Supervisory Signal for Embodied Agents1 Dec 2022
- 2022Predicting Topological Maps for Visual Navigation in Unexplored Environments23 Nov 2022
- 2022Towards Versatile Embodied Navigation30 Oct 2022
- 2022Movement Analytics: Current Status, Application to Manufacturing, and Future Prospects from an AI Perspective4 Oct 2022
- 2022Spatial Relation Graph and Graph Convolutional Network for Object Goal Navigation27 Aug 2022
- 2022A Simple Approach for Visual Rearrangement: 3D Mapping and Semantic Search21 Jun 2022
- 2022Episodic Memory Question Answering3 May 2022
- 2022Continuous Scene Representations for Embodied AI31 Mar 2022
- 2022Reasoning with Scene Graphs for Robot Planning under Partial Observability21 Feb 2022
- 2022Navigating to Objects in Unseen Environments by Distance Prediction8 Feb 2022
- 2022Zero Experience Required: Plug & Play Modular Transfer Learning for Semantic Visual Navigation5 Feb 2022
- 2022SGUIE-Net: Semantic Attention Guided Underwater Image Enhancement with Multi-Scale Perception8 Jan 2022
- 2021Symmetry-aware Neural Architecture for Embodied Visual Navigation17 Dec 2021
- 2021Agent-Centric Relation Graph for Object Visual Navigation29 Nov 2021
- 2021No RL, No Simulation: Learning to Navigate without Navigating18 Oct 2021
- 2021Lifelong Topological Visual Navigation16 Oct 2021
- 2021Learning Efficient Multi-Agent Cooperative Visual Exploration12 Oct 2021
- 2021Hierarchical Object-to-Zone Graph for Object Navigation5 Sep 2021
- 2021Vision-Language Navigation: A Survey and Taxonomy26 Aug 2021
- 2021Hierarchical Representations and Explicit Memory: Learning Effective Navigation Policies on 3D Scene Graphs using Graph Neural Networks2 Aug 2021
- 2021Deep Learning for Embodied Vision Navigation: A Survey7 Jul 2021
- 2021Graph Convolutional Memory using Topological Priors27 Jun 2021
- 2021VTNet: Visual Transformer Network for Object Goal Navigation20 May 2021
- 2021Move2Hear: Active Audio-Visual Source Separation15 May 2021
- 2021Visual Navigation with Spatial Attention20 Apr 2021
- 2021Structured Scene Memory for Vision-Language Navigation5 Mar 2021
- 2021Hierarchical and Partially Observable Goal-driven Policy Learning with Goals Relational Graph1 Mar 2021
- 2021Environment Predictive Coding for Embodied Agents3 Feb 2021
- 2021Memory-Augmented Reinforcement Learning for Image-Goal Navigation13 Jan 2021
- 2021Structured Scene Memory for Vision-Language Navigationcited by 95
- 2022Zero Experience Required: Plug & Play Modular Transfer Learning for Semantic Visual Navigationcited by 66
- 2021Memory-Augmented Reinforcement Learning for Image-Goal Navigationcited by 64
- 2021No RL, No Simulation: Learning to Navigate without Navigatingcited by 63
- 2021VTNet: Visual Transformer Network for Object Goal Navigationcited by 50
- 2021Hierarchical Representations and Explicit Memory: Learning Effective Navigation Policies on 3D Scene Graphs using Graph Neural Networkscited by 50
- 2023Navigating to Objects Specified by Imagescited by 50
- 2021Visual Navigation with Spatial Attentioncited by 43
- 2022Continuous Scene Representations for Embodied AIcited by 39
- 2021Move2Hear: Active Audio-Visual Source Separationcited by 35
- 2021Hierarchical Object-to-Zone Graph for Object Navigationcited by 34
- 2021Vision-Language Navigation: A Survey and Taxonomycited by 33
12 most cited of 45 All 45 (JSON)
- 2026RAO-Nav: Probing Omni-Language Models for Zero-shot Semantic Audio-Visual Navigation26 Sep 2026
- 2026PixelLoop: Shortcut Topological Navigation with Pixel-Level Loops14 Jul 2026
- 2026Semantic Audio-Visual Navigation in Continuous Environments20 Mar 2026
- 2026Memory-Guided View Refinement for Dynamic Human-in-the-loop EQA10 Mar 2026
- 2025ObjectReact: Learning Object-Relative Control for Visual Navigation11 Sep 2025
- 2025Memory-Centric Embodied Question Answering20 May 2025
- 2025UAS Visual Navigation in Large and Unseen Environments via a Meta Agent20 Mar 2025
- 2024Semantic Environment Atlas for Object-Goal Navigation5 Oct 2024
- 2024Aligning Knowledge Graph with Visual Perception for Object-goal Navigation29 Feb 2024
- 2023Selective Visual Representations Improve Convergence and Generalization for Embodied AI7 Nov 2023
- 2023Implicit Obstacle Map-driven Indoor Navigation Model for Robust Obstacle Avoidance24 Aug 2023
- 2023Pseudo-Trilateral Adversarial Training for Domain Adaptive Traversability Prediction26 Jun 2023
12 newest of 45 All 45 (JSON)