mlsanityData to 14 Oct 2026

A Benchmark for Low-Switching-Cost Reinforcement Learning

arXiv 2112.06424 ↗First posted 13 Dec 2021cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

A ubiquitous requirement in many practical reinforcement learning (RL) applications, including medical treatment, recommendation system, education and robotics, is that the deployed policy that actually interacts with the environment cannot change frequently. Such an RL setting is called low-switching-cost RL, i.e., achieving the highest reward while reducing the number of policy switches during training. Despite the recent trend of theoretical studies aiming to design provably efficient RL algorithms with low switching costs, none of the existing approaches have been thoroughly evaluated in popular RL testbeds. In this paper, we systematically studied a wide collection of policy-switching approaches, including theoretically guided criteria, policy-difference-based methods, and non-adaptive baselines. Through extensive experiments on a medical treatment environment, the Atari games, and robotic control tasks, we present the first empirical benchmark for low-switching-cost RL and report novel findings on how to decrease the switching cost while maintain a similar sample efficiency to the case without the low-switching-cost constraint. We hope this benchmark could serve as a starting point for developing more practically effective low-switching-cost RL algorithms. We release our code and complete results in https://sites.google.com/view/low-switching-cost-rl.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 18 of its 28 references that are in the map. First posted on arXiv 13 Dec 2021.

Builds on

18 of 28 references.

References · 18

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  2. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
  3. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,809
  4. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 786
  5. 2016Deep Reinforcement Learning for Robotic Manipulation with Asynchronous Off-Policy Updatescited by 353
  6. 2016#Exploration: A Study of Count-Based Exploration for Deep Reinforcement Learningcited by 309
  7. 2019Way Off-Policy Batch Deep Reinforcement Learning of Implicit Human Preferences in Dialogcited by 284
  8. 2021MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scalecited by 229
  9. 2020Almost Optimal Model-Free Reinforcement Learning via Reference-Advantage Decompositioncited by 127
  10. 2020Deployment-Efficient Reinforcement Learning via Model-Based Offline Optimizationcited by 120
  11. 2020Is Reinforcement Learning More Difficult Than Bandits? A Near-optimal Algorithm Escaping the Curse of Horizoncited by 99
  12. 2017Device Placement Optimization with Reinforcement Learningcited by 86
6 more
  1. 2020Chip Placement with Deep Reinforcement Learningcited by 83
  2. 2019Provably Efficient Q-Learning with Low Switching Costcited by 78
  3. 2013Online Learning with Switching Costs and Other Adaptive Adversariescited by 61
  4. 2020Linear Bandits with Limited Adaptivity and Learning Distributional Optimal Designcited by 51
  5. 2017Applications of Deep Learning and Reinforcement Learning to Biological Datacited by 45
  6. 2021A Provably Efficient Algorithm for Linear Markov Decision Process with Low Switching Costcited by 40

Built on by

Not cited yet.