mlsanityData to 7 Oct 2026

Improved Robustness of Deep Reinforcement Learning for Control of Time-Varying Systems by Bounded Extremum Seeking

arXiv 2510.02490 ↗First posted 2 Oct 2025cs.LG cs.SY eess.SYcited by 3

Not on the map. Why?

Abstract (arXiv)

In this paper, we study the use of robust model independent bounded extremum seeking (ES) feedback control to improve the robustness of deep reinforcement learning (DRL) controllers for a class of nonlinear time-varying systems. DRL has the potential to learn from large datasets to quickly control or optimize the outputs of many-parameter systems, but its performance degrades catastrophically when the system model changes rapidly over time. Bounded ES can handle time-varying systems with unknown control directions, but its convergence speed slows down as the number of tuned parameters increases and, like all local adaptive methods, it can get stuck in local minima. We demonstrate that together, DRL and bounded ES result in a hybrid controller whose performance exceeds the sum of its parts with DRL taking advantage of historical data to learn how to quickly control a many-parameter system to a desired setpoint while bounded ES ensures its robustness to time variations. We present a numerical study of a general time-varying system and a combined ES-DRL controller for automatic tuning of the Low Energy Beam Transport section at the Los Alamos Neutron Science Center linear particle accelerator.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 17 of its 42 references that are in the map. First posted on arXiv 2 Oct 2025.

Builds on

17 of 42 references.

References · 17

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  3. 2015Continuous control with deep reinforcement learningcited by 3,279
  4. 2016RL2: Fast Reinforcement Learning via Slow Reinforcement Learningcited by 490
  5. 2018Multi-Goal Reinforcement Learning: Challenging Robotics Environments and Request for Researchcited by 256
  6. 2015Contextual Markov Decision Processescited by 213
  7. 2024Deep Reinforcement Learning for Robotics: A Survey of Real-World Successescited by 153
  8. 2022Partially Observable Markov Decision Processes in Robotics: A Surveycited by 81
  9. 2021Safe Reinforcement Learning Using Robust Control Barrier Functionscited by 44
  10. 2021CARL: A Benchmark for Contextual and Adaptive Reinforcement Learningcited by 21
  11. 2023Actor-Critic Model Predictive Control: Differentiable Optimization meets Reinforcement Learning for Agile Flightcited by 13
  12. 2020Model-free and Bayesian Ensembling Model-based Deep Reinforcement Learning for Particle Accelerator Control Demonstrated on the FERMI FELcited by 7
5 more
  1. 2024Model-Based Transfer Learning for Contextual Reinforcement Learningcited by 7
  2. 2021Online Policies for Real-Time Control Using MRAC-RLcited by 5
  3. 2024Force Push: Robust Single-Point Pushing with Force Feedbackcited by 3
  4. 2024Predictive control for nonlinear stochastic systems: Closed-loop guarantees with unbounded noisecited by 3
  5. 2022Recursive Averaging with Application to Bio-Inspired 3D Source Seekingcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Planning for Change: Reinforcement Learning Combined with Bounded Extremum Seeking for Robotic Control under Distribution Shiftcited by 2
  2. 2026Mahalanobis-Guided Latent OOD Detection for Hybrid ES-DRL Control in Time-Varying Systemscited by 1
  3. 2026Bidirectional Autoregressive Latent Diffusion for Forward and Inverse Magnetohydrodynamicscited by 1