mlsanityData to 7 Oct 2026

FeUdal Networks for Hierarchical Reinforcement Learning

arXiv 1703.01161 ↗First posted 3 Mar 2017cs.AIcited by 328

Not on the map. Why?

Cited by 34 in Offline RL and imitation learning from data, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We introduce FeUdal Networks (FuNs): a novel architecture for hierarchical reinforcement learning. Our approach is inspired by the feudal reinforcement learning proposal of Dayan and Hinton, and gains power and efficacy by decoupling end-to-end learning across multiple levels -- allowing it to utilise different resolutions of time. Our framework employs a Manager module and a Worker module. The Manager operates at a lower temporal resolution and sets abstract goals which are conveyed to and enacted by the Worker. The Worker generates primitive actions at every tick of the environment. The decoupled structure of FuN conveys several benefits -- in addition to facilitating very long timescale credit assignment it also encourages the emergence of sub-policies associated with different goals set by the Manager. These properties allow FuN to dramatically outperform a strong baseline agent on tasks that involve long-term credit assignment or memorisation. We demonstrate the performance of our proposed system on a range of tasks from the ATARI suite and also from a 3D DeepMind Lab environment.

Key facts. Cited by 328 papers in the map (data through 7 Oct 2026). First posted on arXiv 3 Mar 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2018: 2; 2019: 0; 2020: 3; 2021: 66; 2022: 62; 2023: 54; 2024: 47; 2025: 40; 2026: 54.

328 papers.

Sort:

By area:

  1. 2018Scalable agent alignment via reward modeling: a research directioncited by 438
  2. 2024A Survey on Vision-Language-Action Models for Embodied AIcited by 274
  3. 2024ArCHer: Training Language Model Agents via Hierarchical Multi-Turn RLcited by 212
  4. 2020OPAL: Offline Primitive Discovery for Accelerating Offline Reinforcement Learningcited by 164
  5. 2021BEHAVIOR: Benchmark for Everyday Household Activities in Virtual, Interactive, and Ecological Environmentscited by 161
  6. 2022Exploration in Deep Reinforcement Learning: A Surveycited by 158
  7. 2023HIQL: Offline Goal-Conditioned RL with Latent States as Actionscited by 133
  8. 2023Large Language Models for Robotics: A Surveycited by 132
  9. 2021Goal-Conditioned Reinforcement Learning with Imagined Subgoalscited by 129
  10. 2023Generative agent-based modeling with actions grounded in physical, social, or digital space using Concordiacited by 119
  11. 2021Scalable Evaluation of Multi-Agent Reinforcement Learning with Melting Potcited by 97
  12. 2025Reinforcement Learning with Action Chunkingcited by 97
38 more
  1. 2022Deep Hierarchical Planning from Pixelscited by 96
  2. 2021From Motor Control to Team Play in Simulated Humanoid Footballcited by 87
  3. 2021Bottom-Up Skill Discovery from Unsegmented Demonstrations for Long-Horizon Robot Manipulationcited by 82
  4. 2022BITS: Bi-level Imitation for Traffic Simulationcited by 80
  5. 2021Asymmetric self-play for automatic goal discovery in robotic manipulationcited by 71
  6. 2024Simple Hierarchical Planning with Diffusioncited by 71
  7. 2023SteP: Stacked LLM Policies for Web Actionscited by 66
  8. 2021The Option Keyboard: Combining Skills in Reinforcement Learningcited by 65
  9. 2021Accelerating Robotic Reinforcement Learning via Parameterized Action Primitivescited by 63
  10. 2023LOTUS: Continual Imitation Learning for Robot Manipulation Through Unsupervised Skill Discoverycited by 63
  11. 2023Auto MC-Reward: Automated Dense Reward Design with Large Language Models for Minecraftcited by 60
  12. 2022Unpacking Reward Shaping: Understanding the Benefits of Reward Engineering on Sample Complexitycited by 59
  13. 2021Hierarchical Reinforcement Learning By Discovering Intrinsic Optionscited by 57
  14. 2022Learning Neuro-Symbolic Skills for Bilevel Planningcited by 55
  15. 2022Retrieval-Augmented Reinforcement Learningcited by 53
  16. 2025Uni-cot: Towards Unified Chain-of-Thought Reasoning Across Text and Visioncited by 52
  17. 2021Landmark-Guided Subgoal Generation in Hierarchical Reinforcement Learningcited by 46
  18. 2021TRAIL: Near-Optimal Imitation Learning with Suboptimal Datacited by 39
  19. 2022Modular Lifelong Reinforcement Learning via Neural Compositioncited by 39
  20. 2021Hierarchical Skills for Efficient Explorationcited by 37
  21. 2022Melting Pot 2.0cited by 37
  22. 2022LEAGUE: Guided Skill Learning and Abstraction for Long-Horizon Manipulationcited by 32
  23. 2024Learning Robust Autonomous Navigation and Locomotion for Wheeled-Legged Robotscited by 32
  24. 2021Stochastic Extragradient: General Analysis and Improved Ratescited by 31
  25. 2023Self-optimizing Feature Generation via Categorical Hashing Representation and Hierarchical Reinforcement Crossingcited by 30
  26. 2021Coach-Player Multi-Agent Reinforcement Learning for Dynamic Team Compositioncited by 28
  27. 2021Unsupervised Skill Discovery with Bottleneck Option Learningcited by 28
  28. 2023Deep Laplacian-based Options for Temporally-Extended Explorationcited by 28
  29. 2022How to Reuse and Compose Knowledge for a Lifetime of Tasks: A Survey on Continual Learning and Functional Compositioncited by 27
  30. 2021Probabilistic Mixture-of-Experts for Efficient Deep Reinforcement Learningcited by 26
  31. 2023Train Once, Get a Family: State-Adaptive Balances for Offline-to-Online Reinforcement Learningcited by 26
  32. 2021Flexible Option Learningcited by 24
  33. 2024MaestroMotif: Skill Design from Artificial Intelligence Feedbackcited by 24
  34. 2021Deciding What to Learn: A Rate-Distortion Approachcited by 23
  35. 2022Causality-driven Hierarchical Structure Discovery for Reinforcement Learningcited by 23
  36. 2022Learning Options via Compressioncited by 22
  37. 2023Dynamic-Resolution Model Learning for Object Pile Manipulationcited by 22
  38. 2025Option-aware Temporally Abstracted Value for Offline Goal-Conditioned Reinforcement Learningcited by 22

50 most cited of 328 All 328 (JSON)