mlsanityData to 7 Oct 2026

TempoRL: Learning When to Act

arXiv 2106.05262 ↗First posted 9 Jun 2021cs.LGcited by 25

Not on the map. Why?

Abstract (arXiv)

Reinforcement learning is a powerful approach to learn behaviour through interactions with an environment. However, behaviours are usually learned in a purely reactive fashion, where an appropriate action is selected based on an observation. In this form, it is challenging to learn when it is necessary to execute new decisions. This makes learning inefficient, especially in environments that need various degrees of fine and coarse control. To address this, we propose a proactive setting in which the agent not only selects an action in a state but also for how long to commit to that action. Our TempoRL approach introduces skip connections between states and learns a skip-policy for repeating the same action along these skips. We demonstrate the effectiveness of TempoRL on a variety of traditional and deep RL environments, showing that our approach is capable of learning successful policies up to an order of magnitude faster than vanilla Q-learning.

Key facts. Cited by 25 papers in the map (data through 7 Oct 2026). Builds on 18 of its 40 references that are in the map. First posted on arXiv 9 Jun 2021.

Builds on

18 of 40 references.

References · 18

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2015Continuous control with deep reinforcement learningcited by 3,279
  4. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,704
  5. 2016The Option-Critic Architecturecited by 433
  6. 2019Emergent Tool Use From Multi-Agent Autocurriculacited by 308
  7. 2015Contextual Markov Decision Processescited by 213
  8. 2019Search on the Replay Buffer: Bridging Planning and Reinforcement Learningcited by 188
  9. 2019Planning with Goal-Conditioned Policiescited by 144
  10. 2017When Waiting is not an Option : Learning Options with a Deliberation Costcited by 57
  11. 2017Markov Decision Processes with Continuous Side Informationcited by 54
  12. 2017Learning to Repeat: Fine Grained Action Repetition for Deep Reinforcement Learningcited by 38
6 more
  1. 2016Strategic Attentive Writer for Learning Macro-Actionscited by 29
  2. 2020Control Frequency Adaptation via Action Persistence in Batch Reinforcement Learningcited by 27
  3. 2019The Termination Criticcited by 24
  4. 2018Learning Robust Optionscited by 17
  5. 2019Continuous-Time Markov Decision Processes with Controlled Observationscited by 7
  6. 2017Learning with Options that Terminate Off-Policycited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 6; 2023: 4; 2024: 5; 2025: 4; 2026: 6.

25 papers.

Sort:

By area:

  1. 2022Automated Reinforcement Learning (AutoRL): A Survey and Open Problemscited by 69
  2. 2022Automated Dynamic Algorithm Configurationcited by 37
  3. 2023Model-based Offline Reinforcement Learning with Count-based Conservatismcited by 16
  4. 2024Open the Black Box: Step-based Policy Updates for Temporally-Correlated Episodic Reinforcement Learningcited by 12
  5. 2022Generative Planning for Temporally Coordinated Exploration in Reinforcement Learningcited by 10
  6. 2024When to Sense and Control? A Time-adaptive Approach for Continuous-Time RLcited by 9
  7. 2023Temporal Transfer Learning for Traffic Optimization with Coarse-grained Advisory Autonomycited by 6
  8. 2026SEAR: Sample Efficient Action Chunking Reinforcement Learningcited by 5
  9. 2024Learning Uncertainty-Aware Temporally-Extended Actionscited by 3
  10. 2025Toward Agents That Reason About Their Computationcited by 3
  11. 2022Simultaneously Updating All Persistence Values in Reinforcement Learningcited by 2
  12. 2023Optimizing Attention and Cognitive Control Costs Using Temporally-Layered Architecturescited by 2
13 more
  1. 2022Temporally Extended Successor Representationscited by 1
  2. 2022Temporally Layered Architecture for Adaptive, Distributed and Continuous Controlcited by 1
  3. 2023Actor-Critic with variable time discretization via sustained actionscited by 1
  4. 2024State-Novelty Guided Action Persistence in Deep Reinforcement Learningcited by 1
  5. 2026Contextual Intelligence The Next Leap for Reinforcement Learningcited by 1
  6. 2024Overcoming Slow Decision Frequencies in Continuous Control: Model-Based Sequence Reinforcement Learning for Model-Free Controlcited by 0
  7. 2025Select before Act: Spatially Decoupled Action Repetition for Continuous Controlcited by 0
  8. 2025Improving planning and MBRL with temporally-extended actionscited by 0
  9. 2025Learning to Wait: Synchronizing Agents with the Physical Worldcited by 0
  10. 2026Spectral Alignment in Forward-Backward Representations via Temporal Abstractioncited by 0
  11. 2026Temporal Self-Imitation Learningcited by 0
  12. 2026Let it Cook: Learning to Wait in Sequential Decision Makingcited by 0
  13. 2026Latent Telepathy: Multi-Robot Communication with Self-Supervised Perceptual Latentscited by 0