mlsanityData to 7 Oct 2026

Exploration--Exploitation in MDPs with Options

arXiv 1703.08667 ↗First posted 25 Mar 2017cs.LG stat.MLcited by 19

Not on the map. Why?

Abstract (arXiv)

While a large body of empirical results show that temporally-extended actions and options may significantly affect the learning performance of an agent, the theoretical understanding of how and when options can be beneficial in online reinforcement learning is relatively limited. In this paper, we derive an upper and lower bound on the regret of a variant of UCRL using options. While we first analyze the algorithm in the general case of semi-Markov decision processes (SMDPs), we show how these results can be translated to the specific case of MDPs with options and we illustrate simple scenarios in which the regret of learning with options can be textitprovably much smaller than the regret suffered when learning with primitive actions.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). First posted on arXiv 25 Mar 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 1; 2021: 5; 2022: 6; 2023: 4; 2024: 1; 2025: 1; 2026: 1.

19 papers.

Sort:

By area:

  1. 2021Temporal Abstraction in Reinforcement Learning with the Successor Representationcited by 52
  2. 2023On the Importance of Exploration for Generalization in Reinforcement Learningcited by 30
  3. 2023A Study of Global and Episodic Bonuses for Exploration in Contextual MDPscited by 19
  4. 2020Provable Hierarchical Imitation Learning via EMcited by 10
  5. 2021Adjacency constraint for efficient hierarchical reinforcement learningcited by 10
  6. 2023LLM Augmented Hierarchical Agentscited by 9
  7. 2021Average-Reward Learning and Planning with Optionscited by 8
  8. 2022Logarithmic regret bounds for continuous-time average-reward Markov decision processescited by 7
  9. 2021Interactive Hierarchical Guidance using Languagecited by 5
  10. 2021Provable Hierarchy-Based Meta-Reinforcement Learningcited by 5
  11. 2022Learning Multi-agent Skills for Tabular Reinforcement Learning using Factor Graphscited by 4
  12. 2022Square-root regret bounds for continuous-time episodic Markov decision processescited by 3
7 more
  1. 2022Towards an Interpretable Hierarchical Agent Framework using Semantic Goalscited by 3
  2. 2022Goal Exploration Augmentation via Pre-trained Skills for Sparse-Reward Long-Horizon Goal-Conditioned Reinforcement Learningcited by 2
  3. 2022ASC me to Do Anything: Multi-task Training for Embodied AIcited by 1
  4. 2024Hierarchical Average-Reward Linearly-solvable Markov Decision Processescited by 1
  5. 2023An Option-Dependent Analysis of Regret Minimization Algorithms in Finite-Horizon Semi-Markov Decision Processescited by 0
  6. 2025Realizable Abstractions: Near-Optimal Hierarchical Reinforcement Learningcited by 0
  7. 2026Hierarchical Behaviour Spacescited by 0