# Multi-fidelity Reinforcement Learning Control for Complex Dynamical Systems

arXiv: https://arxiv.org/abs/2504.05588 · HTML: https://mlsanity.com/paper/2504.05588/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods.
- Builds on 22 of its 50 references that are in the map.
- First posted on arXiv 8 Apr 2025.

## Placed in

- Deep reinforcement learning methods / [RL theory and sample complexity](https://mlsanity.com/deeprl/rl-theory-and-sample-complexity/)

## Builds on (most cited first, 22 of 22)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Averaging Weights Leads to Wider Optima and Better Generalization](https://mlsanity.com/paper/1803.05407/) (2018)
- [MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge](https://mlsanity.com/paper/2206.08853/) (2022)
- [Predicting Physics in Mesh-reduced Space with Temporal Attention](https://mlsanity.com/paper/2201.09113/) (2022)
- [Scientific multi-agent reinforcement learning for wall-models of turbulent flows](https://mlsanity.com/paper/2106.11144/) (2021)
- [Multi-resolution partial differential equations preserved learning framework for spatiotemporal dynamics](https://mlsanity.com/paper/2205.03990/) (2022)
- [Learned Turbulence Modelling with Differentiable Fluid Solvers: Physics-based Loss-functions and Optimisation Horizons](https://mlsanity.com/paper/2202.06988/) (2022)
- [Debias Coarsely, Sample Conditionally: Statistical Downscaling through Optimal Transport and Probabilistic Diffusion Models](https://mlsanity.com/paper/2305.15618/) (2023)
- [Control of chaotic systems by Deep Reinforcement Learning](https://mlsanity.com/paper/1906.07672/) (2019)
- [Bayesian Conditional Diffusion Models for Versatile Spatiotemporal Turbulence Generation](https://mlsanity.com/paper/2311.07896/) (2023)
- [Effective control of two-dimensional Rayleigh--B'enard convection: invariant multi-agent reinforcement learning is all you need](https://mlsanity.com/paper/2304.02370/) (2023)
- [SINDy-RL: Interpretable and Efficient Model-Based Reinforcement Learning](https://mlsanity.com/paper/2403.09110/) (2024)
- [Physics-informed Dyna-Style Model-Based Deep Reinforcement Learning for Dynamic Control](https://mlsanity.com/paper/2108.00128/) (2021)
- [Multi-fidelity Generative Deep Learning Turbulent Flows](https://mlsanity.com/paper/2006.04731/) (2020)
- [Multi-fidelity reduced-order surrogate modeling](https://mlsanity.com/paper/2309.00325/) (2023)
- [Multi-fidelity reinforcement learning framework for shape optimization](https://mlsanity.com/paper/2202.11170/) (2022)
- [Decision Stacks: Flexible Reinforcement Learning via Modular Generative Models](https://mlsanity.com/paper/2306.06253/) (2023)
- [Restoring Chaos Using Deep Reinforcement Learning](https://mlsanity.com/paper/1912.00947/) (2019)
- [Multifidelity Reinforcement Learning with Control Variates](https://mlsanity.com/paper/2206.05165/) (2022)
- [Multi-Fidelity Reinforcement Learning with Gaussian Processes](https://mlsanity.com/paper/1712.06489/) (2017)

## Built on by (most cited first, 2 of 2)

- [Progressive multi-fidelity learning with neural networks for physical system predictions](https://mlsanity.com/paper/2510.13762/) (2025)
- [Offline Reinforcement Learning for Fluid Controls: Data-based Multi-observational Policy Extraction](https://mlsanity.com/paper/2606.31025/) (2026)

## Abstract (arXiv)

Controlling instabilities in complex dynamical systems is challenging in scientific and engineering applications. Deep reinforcement learning (DRL) has seen promising results for applications in different scientific applications. The many-query nature of control tasks requires multiple interactions with real environments of the underlying physics. However, it is usually sparse to collect from the experiments or expensive to simulate for complex dynamics. Alternatively, controlling surrogate modeling could mitigate the computational cost issue. However, a fast and accurate learning-based model by offline training makes it very hard to get accurate pointwise dynamics when the dynamics are chaotic. To bridge this gap, the current work proposes a multi-fidelity reinforcement learning (MFRL) framework that leverages differentiable hybrid models for control tasks, where a physics-based hybrid model is corrected by limited high-fidelity data. We also proposed a spectrum-based reward function for RL learning. The effect of the proposed framework is demonstrated on two complex dynamics in physics. The statistics of the MFRL control result match that computed from many-query evaluations of the high-fidelity environments and outperform other SOTA baselines.