mlsanityData to 7 Oct 2026

Object Memory Transformer for Object Goal Navigation

arXiv 2203.14708 ↗First posted 24 Mar 2022cs.CV cs.AI cs.LGcited by 19

Not on the map. Why?

Abstract (arXiv)

This paper presents a reinforcement learning method for object goal navigation (ObjNav) where an agent navigates in 3D indoor environments to reach a target object based on long-term observations of objects and scenes. To this end, we propose Object Memory Transformer (OMT) that consists of two key ideas: 1) Object-Scene Memory (OSM) that enables to store long-term scenes and object semantics, and 2) Transformer that attends to salient objects in the sequence of previously observed scenes and objects stored in OSM. This mechanism allows the agent to efficiently navigate in the indoor environment without prior knowledge about the environments, such as topological maps or 3D meshes. To the best of our knowledge, this is the first work that uses a long-term memory of object semantics in a goal-oriented navigation task. Experimental results conducted on the AI2-THOR dataset show that OMT outperforms previous approaches in navigating in unknown environments. In particular, we show that utilizing the long-term object semantics information improves the efficiency of navigation.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Builds on 23 of its 33 references that are in the map. First posted on arXiv 24 Mar 2022.

Builds on

23 of 33 references.

References · 23

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  4. 2020End-to-End Object Detection with Transformerscited by 7,384
  5. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  6. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  7. 2021Transformers in Vision: A Surveycited by 1,009
  8. 2017AI2-THOR: An Interactive 3D Environment for Visual AIcited by 850
  9. 2014Neural Turing Machinescited by 630
  10. 2018On Evaluation of Embodied Navigation Agentscited by 526
  11. 2016Target-driven Visual Navigation in Indoor Scenes using Deep Reinforcement Learningcited by 351
  12. 2020Agent57: Outperforming the Atari Human Benchmarkcited by 300
11 more
  1. 2018Semi-parametric Topological Memory for Navigationcited by 203
  2. 2017Neural Episodic Controlcited by 148
  3. 2018Visual Semantic Navigation using Scene Priorscited by 127
  4. 2019Scene Memory Transformer for Embodied Agents in Long-Horizon Taskscited by 106
  5. 2018Learning to Learn How to Learn: Self-Adaptive Visual Navigation Using Meta-Learningcited by 91
  6. 2020Learning Object Relation Graph and Tentative Policy for Visual Navigationcited by 62
  7. 2016Control of Memory, Active Perception, and Action in Minecraftcited by 61
  8. 2021VTNet: Visual Transformer Network for Object Goal Navigationcited by 50
  9. 2021Visual Navigation with Spatial Attentioncited by 43
  10. 2017Memory Augmented Control Networkscited by 13
  11. 2018Active Object Perceiver: Recognition-guided Policy Learning for Object Searching on Mobile Robotscited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 9; 2024: 4; 2025: 4; 2026: 1.

19 papers.

Sort:

By area:

  1. 2023Can an Embodied Agent Find Your "Cat-shaped Mug"? LLM-Guided Exploration for Zero-Shot Object Navigationcited by 74
  2. 2025Mem2Ego: Empowering Vision-Language Models with Global-to-Ego Memory for Long-Horizon Embodied Navigationcited by 30
  3. 2022Search for or Navigate to? Dual Adaptive Thinking for Object Navigationcited by 19
  4. 2023Transformers in Reinforcement Learning: A Surveycited by 11
  5. 2023Multiple Thinking Achieving Meta-Ability Decoupling for Object Navigationcited by 8
  6. 2023Switching Head-Tail Funnel UNITER for Dual Referring Expression Comprehension with Fetch-and-Carry Taskscited by 5
  7. 2023Multi-goal Audio-visual Navigation using Sound Direction Mapcited by 5
  8. 2024TDANet: Target-Directed Attention Network For Object-Goal Visual Navigation With Zero-Shot Abilitycited by 5
  9. 2024Select2Plan: Training-Free ICL-Based Planning through VQA and Memory Retrievalcited by 5
  10. 2023TransFusionOdom: Interpretable Transformer-based LiDAR-Inertial Fusion Odometry Estimationcited by 4
  11. 2023Implicit Obstacle Map-driven Indoor Navigation Model for Robust Obstacle Avoidancecited by 3
  12. 2023Building Category Graphs Representation with Spatial and Temporal Attention for Visual Navigationcited by 2
7 more
  1. 2023Learning to Terminate in Object Navigationcited by 1
  2. 2025ImagineNav++: Prompting Vision-Language Models as Embodied Navigator through Scene Imaginationcited by 1
  3. 2026AERR-Nav: Adaptive Exploration-Recovery-Reminiscing Strategy for Zero-Shot Object Navigationcited by 1
  4. 2024Leveraging Unknown Objects to Construct Labeled-Unlabeled Meta-Relationships for Zero-Shot Object Navigationcited by 0
  5. 2024Efficient Strategy Learning by Decoupling Searching and Pathfinding for Object Navigationcited by 0
  6. 2025Collision-Aware Object-Goal Visual Navigation via Two-Stage Deep Reinforcement Learningcited by 0
  7. 2025GRIP: A Unified Framework for Grid-Based Relay and Co-Occurrence-Aware Planning in Dynamic Environmentscited by 0