# Focus on Impact: Indoor Exploration with Intrinsic Motivation

arXiv: https://arxiv.org/abs/2109.08521 · HTML: https://mlsanity.com/paper/2109.08521/

## Key facts

- Cited by 8 papers in the map (data through 7 Oct 2026).
- Builds on 28 of its 41 references that are in the map.
- First posted on arXiv 14 Sep 2021.

## Builds on (most cited first, 28 of 28)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [Matterport3D: Learning from RGB-D Data in Indoor Environments](https://mlsanity.com/paper/1709.06158/) (2017)
- [Curiosity-driven Exploration by Self-supervised Prediction](https://mlsanity.com/paper/1705.05363/) (2017)
- [Habitat: A Platform for Embodied AI Research](https://mlsanity.com/paper/1904.01201/) (2019)
- [Exploration by Random Network Distillation](https://mlsanity.com/paper/1810.12894/) (2018)
- [Conditional Image Generation with PixelCNN Decoders](https://mlsanity.com/paper/1606.05328/) (2016)
- [Image Transformer](https://mlsanity.com/paper/1802.05751/) (2018)
- [Unifying Count-Based Exploration and Intrinsic Motivation](https://mlsanity.com/paper/1606.01868/) (2016)
- [Gibson Env: Real-World Perception for Embodied Agents](https://mlsanity.com/paper/1808.10654/) (2018)
- [DD-PPO: Learning Near-Perfect PointGoal Navigators from 2.5 Billion Frames](https://mlsanity.com/paper/1911.00357/) (2019)
- [Learning to Explore using Active Neural SLAM](https://mlsanity.com/paper/2004.05155/) (2020)
- [Self-Supervised Exploration via Disagreement](https://mlsanity.com/paper/1906.04161/) (2019)
- [Flow++: Improving Flow-Based Generative Models with Variational Dequantization and Architecture Design](https://mlsanity.com/paper/1902.00275/) (2019)
- [Count-Based Exploration with Neural Density Models](https://mlsanity.com/paper/1703.01310/) (2017)
- [RoboTHOR: An Open Simulation-to-Real Embodied AI Platform](https://mlsanity.com/paper/2004.06799/) (2020)
- [RIDE: Rewarding Impact-Driven Exploration for Procedurally-Generated Environments](https://mlsanity.com/paper/2002.12292/) (2020)
- [Learning Exploration Policies for Navigation](https://mlsanity.com/paper/1903.01959/) (2019)
- [Occupancy Anticipation for Efficient Exploration and Navigation](https://mlsanity.com/paper/2008.09285/) (2020)
- [An Exploration of Embodied Visual Exploration](https://mlsanity.com/paper/2001.02192/) (2020)
- [Hierarchical Cross-Modal Agent for Robotics Vision-and-Language Navigation](https://mlsanity.com/paper/2104.10674/) (2021)
- [PyRobot: An Open-source Robotics Framework for Research and Benchmarking](https://mlsanity.com/paper/1906.08236/) (2019)
- [Bi-directional Domain Adaptation for Sim2Real Transfer of Embodied Navigation Agents](https://mlsanity.com/paper/2011.12421/) (2020)
- [BeBold: Exploration Beyond the Boundary of Explored Regions](https://mlsanity.com/paper/2012.08621/) (2020)
- [Embodied Visual Navigation with Automatic Curriculum Learning in Real Environments](https://mlsanity.com/paper/2009.05429/) (2020)
- [Explore and Explain: Self-supervised Navigation and Recounting](https://mlsanity.com/paper/2007.07268/) (2020)
- [Out of the Box: Embodied Navigation in the Real World](https://mlsanity.com/paper/2105.05873/) (2021)

## Built on by (most cited first, 8 of 8)

- [Retrospectives on the Embodied AI Workshop](https://mlsanity.com/paper/2210.06849/) (2022)
- [Off-Policy Evaluation with Online Adaptation for Robot Exploration in Challenging Environments](https://mlsanity.com/paper/2204.03140/) (2022)
- [Embodied Agents for Efficient Exploration and Smart Scene Description](https://mlsanity.com/paper/2301.07150/) (2023)
- [Spot the Difference: A Novel Task for Embodied Agents in Changing Environments](https://mlsanity.com/paper/2204.08502/) (2022)
- [Embodied Navigation at the Art Gallery](https://mlsanity.com/paper/2204.09069/) (2022)
- [Mapping High-level Semantic Regions in Indoor Environments without Object Recognition](https://mlsanity.com/paper/2403.07076/) (2024)
- [Adventurer: Exploration with BiGAN for Deep Reinforcement Learning](https://mlsanity.com/paper/2503.18612/) (2025)
- [Remember to be Curious: Episodic Context and Persistent Worlds for 3D Exploration](https://mlsanity.com/paper/2605.22814/) (2026)

## Abstract (arXiv)

Exploration of indoor environments has recently experienced a significant interest, also thanks to the introduction of deep neural agents built in a hierarchical fashion and trained with Deep Reinforcement Learning (DRL) on simulated environments. Current state-of-the-art methods employ a dense extrinsic reward that requires the complete a priori knowledge of the layout of the training environment to learn an effective exploration policy. However, such information is expensive to gather in terms of time and resources. In this work, we propose to train the model with a purely intrinsic reward signal to guide exploration, which is based on the impact of the robot's actions on its internal representation of the environment. So far, impact-based rewards have been employed for simple tasks and in procedurally generated synthetic environments with countable states. Since the number of states observable by the agent in realistic indoor environments is non-countable, we include a neural-based density model and replace the traditional count-based regularization with an estimated pseudo-count of previously visited states. The proposed exploration approach outperforms DRL-based competitors relying on intrinsic rewards and surpasses the agents trained with a dense extrinsic reward computed with the environment layouts. We also show that a robot equipped with the proposed approach seamlessly adapts to point-goal navigation and real-world deployment.