mlsanityData to 14 Oct 2026

Control Regularization for Reduced Variance Reinforcement Learning

arXiv 1905.05380 ↗First posted 14 May 2019cs.LG cs.SY eess.SYcited by 37

Not on the map. Why?

Abstract (arXiv)

Dealing with high variance is a significant challenge in model-free reinforcement learning (RL). Existing methods are unreliable, exhibiting high variance in performance from run to run using different initializations/seeds. Focusing on problems arising in continuous control, we propose a functional regularization approach to augmenting model-free RL. In particular, we regularize the behavior of the deep policy to be similar to a policy prior, i.e., we regularize in function space. We show that functional regularization yields a bias-variance trade-off, and propose an adaptive tuning strategy to optimize this trade-off. When the policy prior has control-theoretic stability guarantees, we further show that this regularization approximately preserves those stability guarantees throughout learning. We validate our approach empirically on a range of settings, and demonstrate significantly reduced variance, guaranteed dynamic stability, and more efficient learning than deep RL alone.

Key facts. Cited by 37 papers in the map (data through 14 Oct 2026). First posted on arXiv 14 May 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 8; 2022: 6; 2023: 9; 2024: 6; 2025: 3; 2026: 5.

37 papers.

Sort:

By area:

  1. 2021Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learningcited by 371
  2. 2021Contraction Theory for Nonlinear Stability Analysis and Learning-based Control: A Tutorial Overviewcited by 45
  3. 2022LyaNet: A Lyapunov Framework for Training Neural ODEscited by 44
  4. 2021The Stability-Efficiency Dilemma: Investigating Sequence Length Warmup for Training GPT Modelscited by 36
  5. 2023Programmatically Grounded, Compositionally Generalizable Robotic Manipulationcited by 26
  6. 2024A Survey on Large Language Model-Based Social Agents in Game-Theoretic Scenarioscited by 23
  7. 2022Robot Learning of Mobile Manipulation with Reachability Behavior Priorscited by 22
  8. 2021Bayesian Controller Fusion: Leveraging Control Priors in Deep Reinforcement Learning for Roboticscited by 20
  9. 2023Reinforcement Learning with Foundation Priors: Let the Embodied Agent Efficiently Learn on Its Owncited by 19
  10. 2023Physics-Regulated Deep Reinforcement Learning: Invariant Embeddingscited by 10
  11. 2023Beyond Black-Box Advice: Learning-Augmented Algorithms for MDPs with Q-Value Predictionscited by 8
  12. 2022Reinforcement Learning Your Way: Agent Characterization through Policy Regularizationcited by 6
25 more
  1. 2022Equipping Black-Box Policies with Model-Based Advice for Stable Nonlinear Controlcited by 6
  2. 2024Reinforcement Learning with Adaptive Regularization for Safe Control of Critical Systemscited by 6
  3. 2024Contextualized Hybrid Ensemble Q-learning: Learning Fast with Control Priorscited by 5
  4. 2022Reinforcement Learning with Prior Policy Guidance for Motion Planning of Dual-Arm Free-Floating Space Robotcited by 4
  5. 2021A3C-S: Automated Agent Accelerator Co-Search towards Efficient Deep Reinforcement Learningcited by 3
  6. 2023Certifiably Robust Reinforcement Learning through Model-Based Abstract Interpretationcited by 3
  7. 2021Robust Reinforcement Learning under model misspecificationcited by 2
  8. 2024Simplex-enabled Safe Continual Learning Machinecited by 2
  9. 2021Stochastic Intervention for Causal Inference via Reinforcement Learningcited by 1
  10. 2021Coordinate-wise Control Variates for Deep Policy Gradientscited by 1
  11. 2022Barrier Certified Safety Learning Control: When Sum-of-Square Programming Meets Reinforcement Learningcited by 1
  12. 2023Reinforcement Learning-Based Air Traffic Deconflictioncited by 1
  13. 2023Learning-Augmented Decentralized Online Convex Optimization in Networkscited by 1
  14. 2023Stable Modular Control via Contraction Theory for Reinforcement Learningcited by 1
  15. 2023Out-of-Distribution-Aware Electric Vehicle Chargingcited by 1
  16. 2024OGMP: Oracle Guided Multi-mode Policies for Agile and Versatile Robot Controlcited by 1
  17. 2024Shape Arithmetic Expressions: Advancing Scientific Discovery Beyond Closed-Form Equationscited by 1
  18. 2025CHEQ-ing the Box: Safe Variable Impedance Learning for Robotic Polishingcited by 1
  19. 2026Behavior-Consistent Deep Reinforcement Learningcited by 1
  20. 2026Safe and Robust Neural Policy Learning with Statistical Verification for Sim-to-Real Deployment in Roboticscited by 1
  21. 2025Mission-Aligned Learning-Informed Control of Autonomous Systems: Formulation and Foundationscited by 0
  22. 2025Real-DRL: Teach and Learn in Realitycited by 0
  23. 2026Input-Side Variance Suppression under Non-Normal Transient Amplification in Continuous-Control Reinforcement Learningcited by 0
  24. 2026Safe Online Learning via Smooth Safety-Structured Policy Compositioncited by 0
  25. 2026Cognitively-Grounded On-Device Runtime Learning for Ground Robots in Unknown Physical Environmentscited by 0