mlsanityData to 7 Oct 2026

HypeRL: Hypernetwork-Based Reinforcement Learning for Control of Parametrized Dynamical Systems

arXiv 2501.04538 ↗First posted 8 Jan 2025cs.LGcited by 3

Not on the map. Why?

Abstract (arXiv)

In this work, we devise a new, general-purpose reinforcement learning strategy for the optimal control of parametric dynamical systems. Such problems frequently arise in applied sciences and engineering and entail a significant complexity when control and/or state variables are distributed in high-dimensional space or depend on varying parameters. Traditional numerical methods, relying on either iterative minimization algorithms -- exploiting, e.g., the solution of the adjoint problem -- or dynamic programming -- also involving the solution of the Hamilton-Jacobi-Bellman (HJB) equation -- while reliable, often become computationally infeasible. In this paper, we propose HypeRL a deep reinforcement learning (DRL) framework to overcome the limitations shown by traditional methods. HypeRL aims at approximating the optimal control policy directly. Specifically, we employ an actor-critic DRL approach to learn an optimal feedback control strategy that can generalize across the range of variation of the parameters. To effectively learn such optimal control laws for different instances of the parameters, encoding the parameter information into the DRL policy and value function neural networks (NNs) is essential. HypeRL uses two additional NNs, called hypernetworks, to learn the weights and biases of the value function and the policy NNs. In this way, HypeRL effectively embeds the parametric information into the value function and policy. We validate the proposed approach on two parametric control problems, namely (I) a 1D parametric Kuramoto-Sivashinsky equation with in-domain control, and (ii) a navigation problem of particle dynamics in a parametric 2D gyre flow. We show that the knowledge of physical and task-dependent information and the encoding of this information via a hypernetwork, are essential ingredients for learning parameter-dependent control policies.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 36 of its 68 references that are in the map. First posted on arXiv 8 Jan 2025.

Builds on

36 of 68 references.

References · 36

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2015Continuous control with deep reinforcement learningcited by 3,279
  4. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  5. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  6. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,704
  7. 2020Meta-Learning in Neural Networks: A Surveycited by 944
  8. 2015Dueling Network Architectures for Deep Reinforcement Learningcited by 769
  9. 2018Behavioral Cloning from Observationcited by 482
  10. 2017A Brief Survey of Deep Reinforcement Learningcited by 466
  11. 2020Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Surveycited by 388
  12. 2016Deep Reinforcement Learning for Robotic Manipulation with Asynchronous Off-Policy Updatescited by 342
24 more
  1. 2017Deep Reinforcement Learning: An Overviewcited by 327
  2. 2018An Introduction to Deep Reinforcement Learningcited by 201
  3. 2019Mastering Complex Control in MOBA Games with Deep Reinforcement Learningcited by 150
  4. 2018State Representation Learning for Control: An Overviewcited by 125
  5. 2023A Brief Review of Hypernetworks in Deep Learningcited by 102
  6. 2016Playing FPS Games with Deep Reinforcement Learningcited by 95
  7. 2019A Survey of Deep Reinforcement Learning in Video Gamescited by 73
  8. 2023For SALE: State-Action Representation Learning for Deep Reinforcement Learningcited by 73
  9. 2019Deep Meta Functionals for Shape Representationcited by 53
  10. 2020Continual Model-Based Reinforcement Learning with Hypernetworkscited by 41
  11. 2022Hypernetworks in Meta-Reinforcement Learningcited by 39
  12. 2015Towards Vision-Based Deep Reinforcement Learning for Robotic Motion Controlcited by 33
  13. 2021Recomposing the Reinforcement Learning Building Blocks with Hypernetworkscited by 33
  14. 2019Control of chaotic systems by Deep Reinforcement Learningcited by 28
  15. 2021Learning Efficient Navigation in Vortical Flow Fieldscited by 26
  16. 2020Controlling Rayleigh-B'enard convection via Reinforcement Learningcited by 25
  17. 2024SINDy-RL: Interpretable and Efficient Model-Based Reinforcement Learningcited by 24
  18. 2023A Survey on Physics Informed Reinforcement Learning: Review and Open Problemscited by 22
  19. 2023Distributed Control of Partial Differential Equations Using Convolutional Reinforcement Learningcited by 21
  20. 2022Unsupervised Representation Learning in Deep Reinforcement Learning: A Reviewcited by 18
  21. 2021Low Dimensional State Representation Learning with Robotics Priors in Continuous Action Spacescited by 9
  22. 2024Parametric PDE Control with Deep Reinforcement Learning and Differentiable L0-Sparse Polynomial Policiescited by 7
  23. 2021Optimal control of point-to-point navigation in turbulent time-dependent flows using Reinforcement Learningcited by 6
  24. 2023Finite Time Lyapunov Exponent Analysis of Model Predictive Control and Reinforcement Learningcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2025Individualised Treatment Effects Estimation with Composite Treatments and Composite Outcomescited by 1
  2. 2026Flow-aware Optimal Navigation in Unsteady Flows through Reinforcement Learningcited by 0
  3. 2026HypEMBER: Hypernetwork-based Ensemble for Robust Policy Learning of Parametrized Dynamical Systemscited by 0