Actor-Critic with variable time discretization via sustained actions
Not on the map. Why?
Abstract (arXiv)
Reinforcement learning (RL) methods work in discrete time. In order to apply RL to inherently continuous problems like robotic control, a specific time discretization needs to be defined. This is a choice between sparse time control, which may be easier to train, and finer time control, which may allow for better ultimate performance. In this work, we propose SusACER, an off-policy RL algorithm that combines the advantages of different time discretization settings. Initially, it operates with sparse time discretization and gradually switches to a fine one. We analyze the effects of the changing time discretization in robotic control environments: Ant, HalfCheetah, Hopper, and Walker2D. In all cases our proposed algorithm outperforms state of the art.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 18 of its 35 references that are in the map. First posted on arXiv 8 Aug 2023.
Builds on
18 of 35 references.
References · 18
- 2014Adam: A Method for Stochastic Optimizationcited by 36,960
- 2017Proximal Policy Optimization Algorithmscited by 14,441
- 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
- 2015Continuous control with deep reinforcement learningcited by 3,279
- 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
- 2019Solving Rubik's Cube with a Robot Handcited by 699
- 2019Reinforcement Learning in Healthcare: A Surveycited by 308
- 2018The Limits and Potentials of Deep Learning for Roboticscited by 104
- 2019Making Deep Q-learning methods robust to time discretizationcited by 51
- 2017Learning to Repeat: Fine Grained Action Repetition for Deep Reinforcement Learningcited by 38
- 2021Deep Reinforcement Learning for the Control of Robotic Manipulation: A Focussed Mini-Reviewcited by 34
- 2020Control Frequency Adaptation via Action Persistence in Batch Reinforcement Learningcited by 27
6 more
- 2021TempoRL: Learning When to Actcited by 25
- 2021Time Discretization-Invariant Safe Action Repetition for Policy Gradient Methodscited by 21
- 2021Hierarchical Reinforcement Learning with Timed Subgoalscited by 20
- 2021An Analysis of Frame-skipping in Reinforcement Learningcited by 17
- 2021ACERAC: Efficient reinforcement learning in fine time discretizationcited by 4
- 2020A framework for reinforcement learning with autocorrelated actionscited by 3
Built on by
Citing papers per year: 2024: 1.
1 paper.