mlsanityData to 7 Oct 2026

Symmetry-aware Neural Architecture for Embodied Visual Navigation

arXiv 2112.09515 ↗First posted 17 Dec 2021cs.CV cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Visual exploration is a task that seeks to visit all the navigable areas of an environment as quickly as possible. The existing methods employ deep reinforcement learning (RL) as the standard tool for the task. However, they tend to be vulnerable to statistical shifts between the training and test data, resulting in poor generalization over novel environments that are out-of-distribution (OOD) from the training data. In this paper, we attempt to improve the generalization ability by utilizing the inductive biases available for the task. Employing the active neural SLAM (ANS) that learns exploration policies with the advantage actor-critic (A2C) method as the base framework, we first point out that the mappings represented by the actor and the critic should satisfy specific symmetries. We then propose a network design for the actor and the critic to inherently attain these symmetries. Specifically, we use G-convolution instead of the standard convolution and insert the semi-global polar pooling (SGPP) layer, which we newly design in this study, in the last section of the critic network. Experimental results show that our method increases area coverage by 8.1 m2 when trained on the Gibson dataset and tested on the MP3D dataset, establishing the new state-of-the-art.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 53 of its 71 references that are in the map. First posted on arXiv 17 Dec 2021.

Builds on

53 of 71 references.

References · 53

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  2. 2017Mask R-CNNcited by 7,259
  3. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  4. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
  5. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,173
  6. 2019Habitat: A Platform for Embodied AI Researchcited by 1,163
  7. 2016Group Equivariant Convolutional Networkscited by 1,154
  8. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 573
  9. 2018On Evaluation of Embodied Navigation Agentscited by 526
  10. 2021Habitat 2.0: Training Home Assistants to Rearrange their Habitatcited by 501
  11. 2018Gibson Env: Real-World Perception for Embodied Agentscited by 477
  12. 2020Reinforcement Learning with Augmented Datacited by 452
38 more
  1. 2020Object Goal Navigation using Goal-Oriented Semantic Explorationcited by 421
  2. 2019Making Convolutional Networks Shift-Invariant Againcited by 390
  3. 2016Target-driven Visual Navigation in Indoor Scenes using Deep Reinforcement Learningcited by 351
  4. 2020Learning to Explore using Active Neural SLAMcited by 308
  5. 2017Cognitive Mapping and Planning for Visual Navigationcited by 209
  6. 2018Semi-parametric Topological Memory for Navigationcited by 203
  7. 2020Neural Topological SLAM for Visual Navigationcited by 193
  8. 2016Learning to Navigate in Complex Environmentscited by 192
  9. 2019Learning Exploration Policies for Navigationcited by 113
  10. 2019Deep Scale-spaces: Equivariance Over Scalecited by 113
  11. 2019Scale-Equivariant Steerable Networkscited by 110
  12. 2020Occupancy Anticipation for Efficient Exploration and Navigationcited by 104
  13. 2019Look, Listen, and Act: Towards Audio-Visual Embodied Navigationcited by 95
  14. 2020MultiON: Benchmarking Semantic Map Memory using Multi-Object Navigationcited by 95
  15. 2016Exploiting Cyclic Symmetry in Convolutional Neural Networkscited by 91
  16. 2021Auxiliary Tasks and Exploration Enable ObjectNavcited by 73
  17. 2020Automatic Data Augmentation for Generalization in Deep Reinforcement Learningcited by 71
  18. 2020Learning Affordance Landscapes for Interaction Exploration in 3D Environmentscited by 65
  19. 2021Memory-Augmented Reinforcement Learning for Image-Goal Navigationcited by 64
  20. 2020Learning Object Relation Graph and Tentative Policy for Visual Navigationcited by 62
  21. 2020An Exploration of Embodied Visual Explorationcited by 56
  22. 2020Auxiliary Tasks Speed Up Learning PointGoal Navigationcited by 52
  23. 2019Bayesian Relational Memory for Semantic Visual Navigationcited by 48
  24. 2020Semantic Curiosity for Active Visual Learningcited by 44
  25. 2021Visual Navigation with Spatial Attentioncited by 43
  26. 2017Learning to Look Around: Intelligently Exploring Unseen Environments for Unknown Taskscited by 37
  27. 2021Hierarchical Object-to-Zone Graph for Object Navigationcited by 34
  28. 2020Scale Equivariance Improves Siamese Trackingcited by 32
  29. 2019Situational Fusion of Visual Representation for Visual Navigationcited by 28
  30. 2020The general theory of permutation equivarant neural networks and higher order graph variational encoderscited by 28
  31. 2020Trajectory Prediction using Equivariant Continuous Convolutioncited by 26
  32. 2020Scale-covariant and scale-invariant Gaussian derivative networkscited by 25
  33. 2019Benchmarking Classic and Learned Navigation in Complex 3D Environmentscited by 24
  34. 2020EgoMap: Projective mapping and structured egocentric memory for Deep RLcited by 23
  35. 2020Fast Frontier-based Information-driven Autonomous Exploration with an MAVcited by 23
  36. 2021Rotation-Equivariant Deep Learning for Diffusion MRIcited by 20
  37. 2020Group Equivariant Generative Adversarial Networkscited by 19
  38. 2021Learning Efficient Multi-Agent Cooperative Visual Explorationcited by 19

50 most cited of 53 All 53 (JSON)

Built on by

Not cited yet.