# TopoExplore: Homology as an Exploration Signal

arXiv: https://arxiv.org/abs/2607.09971 · HTML: https://mlsanity.com/paper/2607.09971/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 27 of its 48 references that are in the map.
- First posted on arXiv 10 Jul 2026.

## Placed in

- Deep reinforcement learning methods / [Deep RL algorithms and representations](https://mlsanity.com/deeprl/deep-rl-algorithms-and-representations/)

## Builds on (most cited first, 27 of 27)

- [Curiosity-driven Exploration by Self-supervised Prediction](https://mlsanity.com/paper/1705.05363/) (2017)
- [Habitat: A Platform for Embodied AI Research](https://mlsanity.com/paper/1904.01201/) (2019)
- [Exploration by Random Network Distillation](https://mlsanity.com/paper/1810.12894/) (2018)
- [Deep Reinforcement Learning at the Edge of the Statistical Precipice](https://mlsanity.com/paper/2108.13264/) (2021)
- [Diversity is All You Need: Learning Skills without a Reward Function](https://mlsanity.com/paper/1802.06070/) (2018)
- [Unifying Count-Based Exploration and Intrinsic Motivation](https://mlsanity.com/paper/1606.01868/) (2016)
- [Illuminating search spaces by mapping elites](https://mlsanity.com/paper/1504.04909/) (2015)
- [#Exploration: A Study of Count-Based Exploration for Deep Reinforcement Learning](https://mlsanity.com/paper/1611.04717/) (2016)
- [Agent57: Outperforming the Atari Human Benchmark](https://mlsanity.com/paper/2003.13350/) (2020)
- [Large-Scale Study of Curiosity-Driven Learning](https://mlsanity.com/paper/1808.04355/) (2018)
- [First return, then explore](https://mlsanity.com/paper/2004.12919/) (2020)
- [Self-Supervised Exploration via Disagreement](https://mlsanity.com/paper/1906.04161/) (2019)
- [Count-Based Exploration with Neural Density Models](https://mlsanity.com/paper/1703.01310/) (2017)
- [Provably Efficient Maximum Entropy Exploration](https://mlsanity.com/paper/1812.02690/) (2018)
- [Go-Explore: a New Approach for Hard-Exploration Problems](https://mlsanity.com/paper/1901.10995/) (2019)
- [Never Give Up: Learning Directed Exploration Strategies](https://mlsanity.com/paper/2002.06038/) (2020)
- [Semi-parametric Topological Memory for Navigation](https://mlsanity.com/paper/1803.00653/) (2018)
- [Habitat Synthetic Scenes Dataset (HSSD-200): An Analysis of 3D Scene Scale and Realism Tradeoffs for ObjectGoal Navigation](https://mlsanity.com/paper/2306.11290/) (2023)
- [Topological Autoencoders](https://mlsanity.com/paper/1906.00722/) (2019)
- [PersLay: A Neural Network Layer for Persistence Diagrams and New Graph Topological Signatures](https://mlsanity.com/paper/1904.09378/) (2019)
- [Deep Learning with Topological Signatures](https://mlsanity.com/paper/1707.04041/) (2017)
- [MultiON: Benchmarking Semantic Map Memory using Multi-Object Navigation](https://mlsanity.com/paper/2012.03912/) (2020)
- [Learning Montezuma's Revenge from a Single Demonstration](https://mlsanity.com/paper/1812.03381/) (2018)
- [Neural Persistence: A Complexity Measure for Deep Neural Networks Using Algebraic Topology](https://mlsanity.com/paper/1812.09764/) (2018)
- [Ground Slow, Move Fast: A Dual-System Foundation Model for Generalizable Vision-and-Language Navigation](https://mlsanity.com/paper/2512.08186/) (2025)
- [Intelligent Go-Explore: Standing on the Shoulders of Giant Foundation Models](https://mlsanity.com/paper/2405.15143/) (2024)
- [Topological Experience Replay](https://mlsanity.com/paper/2203.15845/) (2022)

## Abstract (arXiv)

Exploration signals in reinforcement learning are currently computed from what an agent has seen: visitation counts, density estimates, or a model's prediction error at individual states. None of these report the holes in the visitation space. To capture them, we compute the persistent homology of the agent's own archive of visited states during exploration and turn each detected class (an enclosed region the archive surrounds but has not entered) into a selection bonus concentrated on the archived states from which entry is possible, gated by an attempt counter that retires candidate entrances and prunes sealed structures. The method, TopoExplore, is Go-Explore plus one additive term, so the comparison with Go-Explore isolates that term. On the 189 held-out worlds of the open-source TopoGym benchmark of topologically varied hard-exploration environments, TopoExplore finds the goal in 167 worlds against 152 for Go-Explore and, where both find it, with a median 126k steps against 186k. On an environment that stress-tests far-apart chambers that create holes in the archive space, TopoExplore enters all six or eight chambers across 5 seeds, against 1 and 2 of 5 seeds for Go-Explore. On Montezuma's Revenge, built over the room graph, the term helps whenever the archive surrounds a room it has not entered, puts 33-45% of the selections on the surrounding rooms while it is active, and enters the room sooner in five of six cases; such enclosures are rare on this game, so the score is unchanged. Beyond discrete, restorable state spaces, the same signal drives an embodied explorer that walks to every target it selects and explores from depth alone; a vision-language-action navigation policy fine-tuned on its trajectories succeeds 58% more often on unseen scenes than one fine-tuned on shortest-path oracle demonstrations.