mlsanityData to 14 Oct 2026

Visual Semantic Navigation using Scene Priors

arXiv 1810.06543 ↗First posted 15 Oct 2018cs.CV cs.AI cs.ROcited by 128

Not on the map. Why?

Abstract (arXiv)

How do humans navigate to target objects in novel scenes? Do we use the semantic/functional priors we have built over years to efficiently search and navigate? For example, to search for mugs, we search cabinets near the coffee machine and for fruits we try the fridge. In this work, we focus on incorporating semantic priors in the task of semantic navigation. We propose to use Graph Convolutional Networks for incorporating the prior knowledge into a deep reinforcement learning framework. The agent uses the features from the knowledge graph to predict the actions. For evaluation, we use the AI2-THOR framework. Our experiments show how semantic knowledge improves performance significantly. More importantly, we show improvement in generalization to unseen scenes and/or objects. The supplementary video can be accessed at the following link: https://youtu.be/otKjuO805dE .

Key facts. Cited by 128 papers in the map (data through 14 Oct 2026). First posted on arXiv 15 Oct 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2020: 1; 2021: 33; 2022: 36; 2023: 24; 2024: 17; 2025: 11; 2026: 6.

128 papers.

Sort:

By area:

  1. 2021A Survey of Embodied AI: From Simulators to Research Taskscited by 261
  2. 2023Diffusion-based Generation, Optimization, and Planning in 3D Scenescited by 239
  3. 2022CoWs on Pasture: Baselines and Benchmarks for Language-Driven Zero-Shot Object Navigationcited by 194
  4. 2022PONI: Potential Functions for ObjectGoal Navigation with Interaction-free Learningcited by 150
  5. 2023ESC: Exploration with Soft Commonsense Constraints for Zero-shot Object Navigationcited by 127
  6. 2024SG-Nav: Online 3D Scene Graph Prompting for LLM-based Zero-shot Object Navigationcited by 109
  7. 2022Navigating to Objects in the Real Worldcited by 106
  8. 2021Visual Room Rearrangementcited by 102
  9. 2021ManipulaTHOR: A Framework for Visual Object Manipulationcited by 102
  10. 2023PIRLNav: Pretraining with Imitation and RL Finetuning for ObjectNavcited by 93
  11. 2021Learning Vision-Guided Quadrupedal Locomotion End-to-End with Cross-Modal Transformerscited by 79
  12. 2023OVRL-V2: A simple state-of-art baseline for ImageNav and ObjectNavcited by 70
38 more
  1. 2022Zero Experience Required: Plug & Play Modular Transfer Learning for Semantic Visual Navigationcited by 67
  2. 2021Memory-Augmented Reinforcement Learning for Image-Goal Navigationcited by 66
  3. 2021No RL, No Simulation: Learning to Navigate without Navigatingcited by 65
  4. 2024PoliFormer: Scaling On-Policy RL with Transformers Results in Masterful Navigatorscited by 58
  5. 2021Semantics for Robotic Mapping, Perception and Interaction: A Surveycited by 56
  6. 2021Hierarchical Representations and Explicit Memory: Learning Effective Navigation Policies on 3D Scene Graphs using Graph Neural Networkscited by 52
  7. 2021VTNet: Visual Transformer Network for Object Goal Navigationcited by 51
  8. 2021Learning to Map for Active Semantic Goal Navigationcited by 50
  9. 2024RoboEXP: Action-Conditioned Scene Graph via Interactive Exploration for Robotic Manipulationcited by 48
  10. 2022Graph Inverse Reinforcement Learning from Diverse Videoscited by 46
  11. 2022Retrospectives on the Embodied AI Workshopcited by 46
  12. 20223D-Aware Object Goal Navigation via Simultaneous Exploration and Identificationcited by 46
  13. 2021Visual Navigation with Spatial Attentioncited by 43
  14. 2021Catch Me If You Hear Me: Audio-Visual Navigation in Complex Unmapped Environments with Moving Soundscited by 41
  15. 2022Continuous Scene Representations for Embodied AIcited by 39
  16. 2021Move2Hear: Active Audio-Visual Source Separationcited by 35
  17. 2021Vision-Language Navigation: A Survey and Taxonomycited by 34
  18. 2021Hierarchical Object-to-Zone Graph for Object Navigationcited by 34
  19. 2022AssistQ: Affordance-centric Question-driven Task Completion for Egocentric Assistantcited by 32
  20. 2022Stubborn: A Strong Baseline for Indoor Object Navigationcited by 32
  21. 2022Zero-shot object goal visual navigationcited by 31
  22. 2021Core Challenges in Embodied Vision-Language Planningcited by 30
  23. 2022TIDEE: Tidying Up Novel Rooms using Visuo-Semantic Commonsense Priorscited by 30
  24. 2021Dimensions of Commonsense Knowledgecited by 29
  25. 2022One Step at a Time: Long-Horizon Vision-and-Language Navigation with Milestonescited by 26
  26. 2022Towards Versatile Embodied Navigationcited by 26
  27. 2022Phone2Proc: Bringing Robust Robots Into Our Chaotic Worldcited by 26
  28. 2021Joint Visual Semantic Reasoning: Multi-Stage Decoder for Text Recognitioncited by 24
  29. 20223D VSG: Long-term Semantic Scene Change Prediction through 3D Variable Scene Graphscited by 24
  30. 2024Empowering Embodied Visual Tracking with Visual Foundation Models and Offline RLcited by 23
  31. 2022PEANUT: Predicting and Navigating to Unseen Targetscited by 22
  32. 2022Learning Active Camera for Multi-Object Navigationcited by 21
  33. 2022Object Memory Transformer for Object Goal Navigationcited by 20
  34. 2022Search for or Navigate to? Dual Adaptive Thinking for Object Navigationcited by 20
  35. 2021Learning Efficient Multi-Agent Cooperative Visual Explorationcited by 19
  36. 2022Unbiased Directed Object Attention Graph for Object Navigationcited by 18
  37. 2023Find What You Want: Learning Demand-conditioned Object Attribute Space for Demand-driven Navigationcited by 18
  38. 2021Shaping embodied agent behavior with activity-context priors from egocentric videocited by 17

50 most cited of 128 All 128 (JSON)