Collision Replay: What Does Bumping Into Things Tell You About Scene Geometry?
Not on the map. Why?
Abstract (arXiv)
What does bumping into things in a scene tell you about scene geometry? In this paper, we investigate the idea of learning from collisions. At the heart of our approach is the idea of collision replay, where we use examples of a collision to provide supervision for observations at a past frame. We use collision replay to train convolutional neural networks to predict a distribution over collision time from new images. This distribution conveys information about the navigational affordances (e.g., corridors vs open spaces) and, as we show, can be converted into the distance function for the scene geometry. We analyze this approach with an agent that has noisy actuation in a photorealistic simulator.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 35 of its 47 references that are in the map. First posted on arXiv 3 May 2021.
Builds on
35 of 47 references.
References · 35
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2014Adam: A Method for Stochastic Optimizationcited by 37,240
- 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
- 2016Feature Pyramid Networks for Object Detectioncited by 5,036
- 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,689
- 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,041
- 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,495
- 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,376
- 2019Habitat: A Platform for Embodied AI Researchcited by 1,191
- 2016Unsupervised Monocular Depth Estimation with Left-Right Consistencycited by 820
- 2019PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitizationcited by 818
- 2017Unsupervised Learning of Depth and Ego-Motion from Videocited by 782
23 more
- 2014Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-Scale Convolutional Architecturecited by 483
- 2018Gibson Env: Real-World Perception for Embodied Agentscited by 481
- 2016Past, Present, and Future of Simultaneous Localization And Mapping: Towards the Robust-Perception Agecited by 411
- 2019DISN: Deep Implicit Surface Network for High-quality Single-view 3D Reconstructioncited by 333
- 2020Learning to Explore using Active Neural SLAMcited by 316
- 2017Cognitive Mapping and Planning for Visual Navigationcited by 212
- 2015Supersizing Self-supervision: Learning to Grasp from 50K Tries and 700 Robot Hourscited by 209
- 2017Multi-view Supervision for Single-view Reconstruction via Differentiable Ray Consistencycited by 177
- 2016Learning to Poke by Poking: Experiential Learning of Intuitive Physicscited by 154
- 2018Visual Semantic Navigation using Scene Priorscited by 128
- 2019Self-Supervised Monocular Depth Hintscited by 127
- 2020Occupancy Anticipation for Efficient Exploration and Navigationcited by 106
- 2020BADGR: An Autonomous Self-Supervised Learning-Based Navigation Systemcited by 102
- 2017Self-supervised Learning of Motion Capturecited by 97
- 2018LayoutNet: Reconstructing the 3D Room Layout from a Single RGB Imagecited by 95
- 2020VisualEchoes: Spatial Image Representation Learning through Echolocationcited by 55
- 2018Talk the Walk: Navigating New York City through Grounded Dialoguecited by 44
- 2017Learning to Fly by Crashingcited by 41
- 2019PyRobot: An Open-source Robotics Framework for Research and Benchmarkingcited by 39
- 2017RoomNet: End-to-End Room Layout Estimationcited by 38
- 2018Active Neural Localizationcited by 18
- 2018A Comparative Study of Bug Algorithms for Robot Navigationcited by 8
- 2019Bounce and Learn: Modeling Scene Dynamics with Real-World Bouncescited by 7
Built on by
Not cited yet.