Symmetry-aware Neural Architecture for Embodied Visual Navigation
Not on the map. Why?
Abstract (arXiv)
Visual exploration is a task that seeks to visit all the navigable areas of an environment as quickly as possible. The existing methods employ deep reinforcement learning (RL) as the standard tool for the task. However, they tend to be vulnerable to statistical shifts between the training and test data, resulting in poor generalization over novel environments that are out-of-distribution (OOD) from the training data. In this paper, we attempt to improve the generalization ability by utilizing the inductive biases available for the task. Employing the active neural SLAM (ANS) that learns exploration policies with the advantage actor-critic (A2C) method as the base framework, we first point out that the mappings represented by the actor and the critic should satisfy specific symmetries. We then propose a network design for the actor and the critic to inherently attain these symmetries. Specifically, we use G-convolution instead of the standard convolution and insert the semi-global polar pooling (SGPP) layer, which we newly design in this study, in the last section of the critic network. Experimental results show that our method increases area coverage by 8.1 m2 when trained on the Gibson dataset and tested on the MP3D dataset, establishing the new state-of-the-art.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 53 of its 71 references that are in the map. First posted on arXiv 17 Dec 2021.
Builds on
53 of 71 references.
References · 53
- 2014Microsoft COCO: Common Objects in Contextcited by 17,498
- 2017Mask R-CNNcited by 7,259
- 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
- 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
- 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,173
- 2019Habitat: A Platform for Embodied AI Researchcited by 1,163
- 2016Group Equivariant Convolutional Networkscited by 1,154
- 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 573
- 2018On Evaluation of Embodied Navigation Agentscited by 526
- 2021Habitat 2.0: Training Home Assistants to Rearrange their Habitatcited by 501
- 2018Gibson Env: Real-World Perception for Embodied Agentscited by 477
- 2020Reinforcement Learning with Augmented Datacited by 452
38 more
- 2020Object Goal Navigation using Goal-Oriented Semantic Explorationcited by 421
- 2019Making Convolutional Networks Shift-Invariant Againcited by 390
- 2016Target-driven Visual Navigation in Indoor Scenes using Deep Reinforcement Learningcited by 351
- 2020Learning to Explore using Active Neural SLAMcited by 308
- 2017Cognitive Mapping and Planning for Visual Navigationcited by 209
- 2018Semi-parametric Topological Memory for Navigationcited by 203
- 2020Neural Topological SLAM for Visual Navigationcited by 193
- 2016Learning to Navigate in Complex Environmentscited by 192
- 2019Learning Exploration Policies for Navigationcited by 113
- 2019Deep Scale-spaces: Equivariance Over Scalecited by 113
- 2019Scale-Equivariant Steerable Networkscited by 110
- 2020Occupancy Anticipation for Efficient Exploration and Navigationcited by 104
- 2019Look, Listen, and Act: Towards Audio-Visual Embodied Navigationcited by 95
- 2020MultiON: Benchmarking Semantic Map Memory using Multi-Object Navigationcited by 95
- 2016Exploiting Cyclic Symmetry in Convolutional Neural Networkscited by 91
- 2021Auxiliary Tasks and Exploration Enable ObjectNavcited by 73
- 2020Automatic Data Augmentation for Generalization in Deep Reinforcement Learningcited by 71
- 2020Learning Affordance Landscapes for Interaction Exploration in 3D Environmentscited by 65
- 2021Memory-Augmented Reinforcement Learning for Image-Goal Navigationcited by 64
- 2020Learning Object Relation Graph and Tentative Policy for Visual Navigationcited by 62
- 2020An Exploration of Embodied Visual Explorationcited by 56
- 2020Auxiliary Tasks Speed Up Learning PointGoal Navigationcited by 52
- 2019Bayesian Relational Memory for Semantic Visual Navigationcited by 48
- 2020Semantic Curiosity for Active Visual Learningcited by 44
- 2021Visual Navigation with Spatial Attentioncited by 43
- 2017Learning to Look Around: Intelligently Exploring Unseen Environments for Unknown Taskscited by 37
- 2021Hierarchical Object-to-Zone Graph for Object Navigationcited by 34
- 2020Scale Equivariance Improves Siamese Trackingcited by 32
- 2019Situational Fusion of Visual Representation for Visual Navigationcited by 28
- 2020The general theory of permutation equivarant neural networks and higher order graph variational encoderscited by 28
- 2020Trajectory Prediction using Equivariant Continuous Convolutioncited by 26
- 2020Scale-covariant and scale-invariant Gaussian derivative networkscited by 25
- 2019Benchmarking Classic and Learned Navigation in Complex 3D Environmentscited by 24
- 2020EgoMap: Projective mapping and structured egocentric memory for Deep RLcited by 23
- 2020Fast Frontier-based Information-driven Autonomous Exploration with an MAVcited by 23
- 2021Rotation-Equivariant Deep Learning for Diffusion MRIcited by 20
- 2020Group Equivariant Generative Adversarial Networkscited by 19
- 2021Learning Efficient Multi-Agent Cooperative Visual Explorationcited by 19
50 most cited of 53 All 53 (JSON)
Built on by
Not cited yet.