Data to 7 Oct 2026

Stochastic Neural Networks for Hierarchical Reinforcement Learning

arXiv 1704.03012 ↗First posted 10 Apr 2017cs.AI cs.LG cs.NEcited by 124

Not on the map. Why?

Abstract (arXiv)

Deep reinforcement learning has achieved many impressive results in recent years. However, tasks with sparse rewards or long horizons continue to pose significant challenges. To tackle these important problems, we propose a general framework that first learns useful skills in a pre-training environment, and then leverages the acquired skills for learning faster in downstream tasks. Our approach brings together some of the strengths of intrinsic motivation and hierarchical methods: the learning of useful skill is guided by a single proxy reward, the design of which requires very minimal domain knowledge about the downstream tasks. Then a high-level policy is trained on top of these skills, providing a significant improvement of the exploration and allowing to tackle sparse rewards in the downstream tasks. To efficiently pre-train a large span of skills, we use Stochastic Neural Networks combined with an information-theoretic regularizer. Our experiments show that this combination is effective in learning a wide span of interpretable skills in a sample-efficient way, and can significantly boost the learning performance uniformly across a wide range of downstream tasks.

Key facts. Cited by 124 papers in the map (data through 7 Oct 2026). First posted on arXiv 10 Apr 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2018: 1; 2019: 1; 2020: 0; 2021: 41; 2022: 32; 2023: 18; 2024: 16; 2025: 10; 2026: 5.

124 papers.

Sort:

By area:

2021PEBBLE: Feedback-Efficient Interactive Reinforcement Learning via Relabeling Experience and Unsupervised Pre-trainingcited by 298
2019Goal-conditioned Imitation Learningcited by 159
2022Deep Hierarchical Planning from Pixelscited by 96
2021Reward is enough for convex MDPscited by 82
2023Modular Deep Learningcited by 82
2023METRA: Scalable Unsupervised RL with Metric-Aware Abstractioncited by 80
2022CIC: Contrastive Intrinsic Control for Unsupervised Skill Discoverycited by 66
2021Hierarchical Reinforcement Learning By Discovering Intrinsic Optionscited by 57
2022Retrieval-Augmented Reinforcement Learningcited by 53
2021Landmark-Guided Subgoal Generation in Hierarchical Reinforcement Learningcited by 46
2021Hierarchical Skills for Efficient Explorationcited by 37
2021The Information Geometry of Unsupervised Reinforcement Learningcited by 36

38 more

2023Self-optimizing Feature Generation via Categorical Hashing Representation and Hierarchical Reinforcement Crossingcited by 30
2021Unsupervised Skill Discovery with Bottleneck Option Learningcited by 28
2021Learning Transferable Motor Skills with Hierarchical Latent Mixture Policiescited by 26
2023Behavior Contrastive Learning for Unsupervised Skill Discoverycited by 26
2021Value Function Spaces: Skill-Centric State Abstractions for Long-Horizon Reasoningcited by 25
2022An information-theoretic perspective on intrinsic motivation in reinforcement learning: a surveycited by 25
2021Diverse Auto-Curriculum is Critical for Successful Real-World Multiagent Learning Systemscited by 24
2022Causality-driven Hierarchical Structure Discovery for Reinforcement Learningcited by 23
2021Pick Your Battles: Interaction Graphs as Population-Level Objectives for Strategic Diversitycited by 22
2022Leveraging Approximate Symbolic Models for Reinforcement Learning via Skill Diversitycited by 22
2022Learning Options via Compressioncited by 22
2021Beyond Fine-Tuning: Transferring Behavior in Reinforcement Learningcited by 21
2022CompoSuite: A Compositional Reinforcement Learning Benchmarkcited by 21
2021Hierarchical Reinforcement Learning with Timed Subgoalscited by 20
2021Solving Compositional Reinforcement Learning Problems via Task Reductioncited by 19
2021Direct then Diffuse: Incremental Unsupervised Skill Discovery for State Covering and Goal Reachingcited by 19
2021Discovering Diverse Solutions in Deep Reinforcement Learning by Maximizing State-Action-Based Mutual Informationcited by 18
2021AMP: Adversarial Motion Priors for Stylized Physics-Based Character Controlcited by 18
2024Can a MISL Fly? Analysis and Ingredients for Mutual Information Skill Learningcited by 18
2021Collect & Infer -- a fresh look at data-efficient Reinforcement Learningcited by 17
2021Composable Energy Policies for Reactive Motion Generation and Reinforcement Learningcited by 16
2021Self-supervised Reinforcement Learning with Independently Controllable Subgoalscited by 16
2023Structure in Deep Reinforcement Learning: A Survey and Open Problemscited by 16
2021Policy Information Capacity: Information-Theoretic Measure for Task Complexity in Deep Reinforcement Learningcited by 15
2021Variational Empowerment as Representation Learning for Goal-Based Reinforcement Learningcited by 15
2021RLOps: Development Life-cycle of Reinforcement Learning Aided Open RANcited by 15
2023Multi-task Hierarchical Adversarial Inverse Reinforcement Learningcited by 15
2024Continuous Control with Coarse-to-fine Reinforcement Learningcited by 15
2023Diverse Conventions for Human-AI Collaborationcited by 14
2021Motion Planning by Learning the Solution Manifold in Trajectory Optimizationcited by 13
2022Versatile Skill Control via Self-supervised Adversarial Imitation of Unlabeled Mixed Motionscited by 13
2024Leveraging Skills from Unlabeled Prior Data for Efficient Online Explorationcited by 13
2021Adaptable Agent Populations via a Generative Model of Policiescited by 12
2021Braxlines: Fast and Interactive Toolkit for RL-driven Behavior Engineering beyond Reward Maximizationcited by 11
2023NetHack is Hard to Hackcited by 11
2024ALaRM: Align Language Models via Hierarchical Rewards Modelingcited by 11
2021Adjacency constraint for efficient hierarchical reinforcement learningcited by 10
2022Generative Planning for Temporally Coordinated Exploration in Reinforcement Learningcited by 10

50 most cited of 124 All 124 (JSON)