mlsanityData to 7 Oct 2026

When Waiting is not an Option : Learning Options with a Deliberation Cost

arXiv 1709.04571 ↗First posted 14 Sep 2017cs.AIcited by 57

Not on the map. Why?

Abstract (arXiv)

Recent work has shown that temporally extended actions (options) can be learned fully end-to-end as opposed to being specified in advance. While the problem of "how" to learn options is increasingly well understood, the question of "what" good options should be has remained elusive. We formulate our answer to what "good" options should be in the bounded rationality framework (Simon, 1957) through the notion of deliberation cost. We then derive practical gradient-based learning algorithms to implement this objective. Our results in the Arcade Learning Environment (ALE) show increased performance and interpretability.

Key facts. Cited by 57 papers in the map (data through 7 Oct 2026). First posted on arXiv 14 Sep 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 11; 2022: 14; 2023: 7; 2024: 9; 2025: 9; 2026: 7.

57 papers.

Sort:

By area:

  1. 2024Code as Reward: Empowering Reinforcement Learning with VLMscited by 38
  2. 2023Deep Laplacian-based Options for Temporally-Extended Explorationcited by 28
  3. 2021Discovery of Options via Meta-Learned Subgoalscited by 27
  4. 2021TempoRL: Learning When to Actcited by 25
  5. 2021Flexible Option Learningcited by 24
  6. 2024MaestroMotif: Skill Design from Artificial Intelligence Feedbackcited by 24
  7. 2021Deciding What to Learn: A Rate-Distortion Approachcited by 23
  8. 2022Learning Options via Compressioncited by 22
  9. 2021Hierarchical Reinforcement Learning with Timed Subgoalscited by 20
  10. 2021HAVEN: Hierarchical Cooperative Multi-Agent Reinforcement Learning with Dual Coordination Mechanismcited by 19
  11. 2022Humans decompose tasks by trading off utility and computational costcited by 18
  12. 2022Reward-Respecting Subtasks for Model-Based Reinforcement Learningcited by 15
38 more
  1. 2021Temporally Abstract Partial Modelscited by 11
  2. 2022Generalised Policy Improvement with Geometric Policy Compositioncited by 11
  3. 2022Dynamic Decision Frequency with Continuous Optionscited by 11
  4. 2023Hierarchical Reinforcement Learning for Power Network Topology Controlcited by 11
  5. 2022MO2: Model-Based Offline Optionscited by 10
  6. 2023Scalable Multi-agent Covering Option Discovery based on Kronecker Graphscited by 9
  7. 2024When to Sense and Control? A Time-adaptive Approach for Continuous-Time RLcited by 9
  8. 2024SkiLD: Unsupervised Skill Discovery Guided by Factor Interactionscited by 9
  9. 2022Attention Option-Criticcited by 8
  10. 2022Leveraging Sequentiality in Reinforcement Learning from a Single Demonstrationcited by 6
  11. 2023Granger Causal Interaction Skill Chainscited by 5
  12. 2025Scalable Option Learning in High-Throughput Environmentscited by 5
  13. 2022Learning Multi-agent Skills for Tabular Reinforcement Learning using Factor Graphscited by 4
  14. 2025Hierarchical Multi-Agent Reinforcement Learning with Control Barrier Functions for Safety-Critical Autonomous Systemscited by 4
  15. 2022Priors, Hierarchy, and Information Asymmetry for Skill Transfer in Reinforcement Learningcited by 3
  16. 2022Toward Discovering Options that Achieve Faster Planningcited by 3
  17. 2024PIPER: Primitive-Informed Preference-based Hierarchical Reinforcement Learning via Hindsight Relabelingcited by 3
  18. 2025Toward Agents That Reason About Their Computationcited by 3
  19. 2021Learning Skills to Navigate without a Master: A Sequential Multi-Policy Reinforcement Learning Algorithmcited by 2
  20. 2023CRISP: Curriculum Inducing Primitive Informed Subgoal Prediction for Hierarchical Reinforcement Learningcited by 2
  21. 2023PEAR: Primitive Enabled Adaptive Relabeling for Boosting Hierarchical Reinforcement Learningcited by 2
  22. 2023Adaptive trajectory-constrained exploration strategy for deep reinforcement learningcited by 2
  23. 2025Behaviour Discovery and Attribution for Explainable Reinforcement Learningcited by 2
  24. 2026Temporally Extended Mixture-of-Experts Modelscited by 2
  25. 2021Reducing the Deployment-Time Inference Control Costs of Deep Reinforcement Learning Agents via an Asymmetric Architecturecited by 1
  26. 2021Optimization for Master-UAV-powered Auxiliary-Aerial-IRS-assisted IoT Networks: An Option-based Multi-agent Hierarchical Deep Reinforcement Learning Approachcited by 1
  27. 2024MENTOR: Guiding Hierarchical Reinforcement Learning with Human Feedback and Dynamic Distance Constraintcited by 1
  28. 2024DIPPER: Direct Preference Optimization to Accelerate Primitive-Enabled Hierarchical Reinforcement Learningcited by 1
  29. 2024Direct Preference Optimization for Primitive-Enabled Hierarchical RL: A Bilevel Approachcited by 1
  30. 2024Subgoal Discovery Using a Free Energy Paradigm and State Aggregationscited by 1
  31. 2025Combining Bayesian Inference and Reinforcement Learning for Agent Decision Making: A Reviewcited by 1
  32. 2021Interpretable Reinforcement Learning Inspired by Piaget's Theory of Cognitive Developmentcited by 0
  33. 2022Multi-agent Deep Covering Skill Discoverycited by 0
  34. 2022Reusable Options through Gradient-based Meta Learningcited by 0
  35. 2022SHIRO: Soft Hierarchical Reinforcement Learningcited by 0
  36. 2025Autonomous Curriculum Design via Relative Entropy Based Task Modificationscited by 0
  37. 2025Learning Temporal Abstractions via Variational Homomorphisms in Option-Induced Abstract MDPscited by 0
  38. 2025Towards General Computer Control with Hierarchical Agents and Multi-Level Action Spacescited by 0

50 most cited of 57 All 57 (JSON)