Hierarchical Representation Learning for Markov Decision Processes
Not on the map. Why?
Abstract (arXiv)
In this paper we present a novel method for learning hierarchical representations of Markov decision processes. Our method works by partitioning the state space into subsets, and defines subtasks for performing transitions between the partitions. We formulate the problem of partitioning the state space as an optimization problem that can be solved using gradient descent given a set of sampled trajectories, making our method suitable for high-dimensional problems with large state spaces. We empirically validate the method, by showing that it can successfully learn a useful hierarchical representation in a navigation domain. Once learned, the hierarchical representation can be used to solve different tasks in the given domain, thus generalizing knowledge across tasks.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 22 of its 49 references that are in the map. First posted on arXiv 3 Jun 2021.
Builds on
22 of 49 references.
References · 22
- 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
- 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,258
- 2015Deep Reinforcement Learning with Double Q-learningcited by 1,745
- 2017Hindsight Experience Replaycited by 1,136
- 2015Prioritized Experience Replaycited by 1,099
- 2018Data-Efficient Hierarchical Reinforcement Learningcited by 455
- 2016The Option-Critic Architecturecited by 448
- 2017Automatic Goal Generation for Reinforcement Learning Agentscited by 259
- 2019Go-Explore: a New Approach for Hard-Exploration Problemscited by 212
- 2019Search on the Replay Buffer: Bridging Planning and Reinforcement Learningcited by 194
- 2015Action-Conditional Video Prediction using Deep Networks in Atari Gamescited by 172
- 2015Learning Continuous Control Policies by Stochastic Value Gradientscited by 162
10 more
- 2018Self-Imitation Learningcited by 138
- 2017A Laplacian Framework for Option Discovery in Reinforcement Learningcited by 110
- 2018Near-Optimal Representation Learning for Hierarchical Reinforcement Learningcited by 110
- 2018Learning Actionable Representations with Goal-Conditioned Policiescited by 54
- 2017Multi-Level Discovery of Deep Optionscited by 51
- 2013On the Complexity of Solving Markov Decision Problemscited by 31
- 2021Discovery of Options via Meta-Learned Subgoalscited by 27
- 2018Efficient Model-Based Deep Reinforcement Learning with Variational State Tabulationcited by 20
- 2019Learning World Graphs to Accelerate Hierarchical Reinforcement Learningcited by 12
- 2016Option Discovery in Hierarchical Reinforcement Learning using Spatio-Temporal Clusteringcited by 6
Built on by
Citing papers per year: 2022: 1; 2023: 2.
3 papers.