mlsanityData to 14 Oct 2026

Physics-enhanced reinforcement learning for real-time optimal control of dynamical systems

arXiv 2607.16177 ↗First posted 17 Jul 2026cs.LG math.OCcited by 0

On the map: Deep reinforcement learning methods → RL theory and sample complexity

Abstract (arXiv)

Reinforcement learning (RL) has recently emerged as a promising feedback control strategy for nonlinear and complex dynamical systems. However, RL algorithms are sample inefficient and require a large number of interaction with the environment to synthesize optimal control strategies. Consequently, applications of RL are typically limited to sparse sensors and actuators due to the curse of dimensionality entailed by the exploration-exploitation dilemma in high-dimensional spaces. In this work, we bridge RL and traditional optimal control for dynamical system with a novel Physics-EnhAnced Reinforcement Learning (PEARL) paradigm tailored to the control of high-dimensional and parametric dynamical systems, exploiting the differentibility of their dynamics. Specifically, PEARL employs an actor-adjoint algorithm that leverages automatic differentiation to compute policy gradients over short horizons and adjoint-based sensitivities of future returns approximated via neural networks, significantly reducing the number of environment interactions, while mitigating long-term gradient instabilities. Through two challenging parametric navigation problems in unsteady flows, we show that PEARL (i) effectively exploits differentiable environments to outperform state-of-the-art RL algorithms, (ii) is sample efficient, thanks to the physics-guided policy learning, (iii) generalizes across multiple scenarios, which is crucial when dealing with parametric systems, and (iv) enables scaling RL to high-dimensional state and action spaces, without requiring low-dimensional state representations or multi-agent strategies.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods. Builds on 47 of its 109 references that are in the map. First posted on arXiv 17 Jul 2026.

Builds on

47 of 109 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024A Pontryagin Perspective on Reinforcement Learning28 May
  2. 2024SINDy-RL: Interpretable and Efficient Model-Based Reinforcement Learning14 Mar

Other references · 45

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  3. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  4. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 2,026
  5. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,745
  6. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
  7. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,264
  8. 2015Automatic differentiation in machine learning: a surveycited by 855
  9. 2019When to Trust Your Model: Model-Based Policy Optimizationcited by 773
  10. 2017Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithmcited by 734
  11. 2013Stochastic First- and Zeroth-order Methods for Nonconvex Stochastic Programmingcited by 724
  12. 2020Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Surveycited by 413
33 more
  1. 2021Brax -- A Differentiable Physics Engine for Large Scale Rigid Body Simulationcited by 407
  2. 2016Deep Reinforcement Learning for Robotic Manipulation with Asynchronous Off-Policy Updatescited by 353
  3. 2017Deep Reinforcement Learning: An Overviewcited by 333
  4. 2018An Introduction to Deep Reinforcement Learningcited by 206
  5. 2015Multiagent Cooperation and Competition with Deep Reinforcement Learningcited by 184
  6. 2018A Survey and Critique of Multiagent Deep Reinforcement Learningcited by 177
  7. 2024Deep Reinforcement Learning for Robotics: A Survey of Real-World Successescited by 176
  8. 2020Learning to Control PDEs with Differentiable Physicscited by 132
  9. 2022Accelerated Policy Learning with Parallel Differentiable Simulationcited by 104
  10. 2016Playing FPS Games with Deep Reinforcement Learningcited by 95
  11. 2022Do Differentiable Simulators Give Better Policy Gradients?cited by 89
  12. 2018Efficient collective swimming by harnessing vortices through deep reinforcement learningcited by 73
  13. 2019A Survey of Deep Reinforcement Learning in Video Gamescited by 73
  14. 2018Variance Reduction for Policy Gradient with Action-Dependent Factorized Baselinescited by 68
  15. 2019A review on Deep Reinforcement Learning for Fluid Mechanicscited by 66
  16. 2022A Survey on Model-based Reinforcement Learningcited by 66
  17. 2019Pontryagin Differentiable Programming: An End-to-End Learning and Control Frameworkcited by 52
  18. 2018Deep Reinforcement Learning for Swarm Systemscited by 45
  19. 2015Towards Vision-Based Deep Reinforcement Learning for Robotic Motion Controlcited by 33
  20. 2019Control of chaotic systems by Deep Reinforcement Learningcited by 29
  21. 2021Implicit energy regularization of neural ordinary-differential-equation controlcited by 29
  22. 2023Effective control of two-dimensional Rayleigh--B'enard convection: invariant multi-agent reinforcement learning is all you needcited by 27
  23. 2020Controlling Rayleigh-B'enard convection via Reinforcement Learningcited by 25
  24. 2023Distributed Control of Partial Differential Equations Using Convolutional Reinforcement Learningcited by 24
  25. 2023A Survey on Physics Informed Reinforcement Learning: Review and Open Problemscited by 24
  26. 2021Physics-informed Dyna-Style Model-Based Deep Reinforcement Learning for Dynamic Controlcited by 23
  27. 2022Training Efficient Controllers via Analytic Policy Gradientcited by 22
  28. 2022Unsupervised Representation Learning in Deep Reinforcement Learning: A Reviewcited by 18
  29. 2021Optimal control of point-to-point navigation in turbulent time-dependent flows using Reinforcement Learningcited by 6
  30. 2024An experimental evaluation of Deep Reinforcement Learning algorithms for HVAC controlcited by 4
  31. 2024Stealing That Free Lunch: Exposing the Limits of Dyna-Style Reinforcement Learningcited by 4
  32. 2025Optimal Navigation in Microfluidics via the Optimization of a Discrete Losscited by 4
  33. 2022Empowering Optimal Control with Machine Learning: A Perspective from Model Predictive Controlcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Physics-Informed Policy Optimization via Analytic Dynamics Regularization
  2. 2024Physics-Informed Model and Hybrid Planning for Efficient Dyna-Style Reinforcement Learning
  3. 2024A Differential and Pointwise Control Approach to Reinforcement Learning
  4. 2026Physics-informed Reinforcement Learning for Stochastic Reach-Avoid Analysis
  5. 2024Physics-Guided Actor-Critic Reinforcement Learning for Swimming in Turbulence
  6. 2025HypeRL: Hypernetwork-Based Reinforcement Learning for Control of Parametrized Dynamical Systems
  7. 2025PEARL: Preconditioner Enhancement through Actor-critic Reinforcement Learning
  8. 2025A critical assessment of reinforcement learning methods for microswimmer navigation in complex flows