# Controlling Rayleigh-B'enard convection via Reinforcement Learning

arXiv: https://arxiv.org/abs/2003.14358 · HTML: https://mlsanity.com/paper/2003.14358/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Mar 2020.

## Built on by (most cited first, 25 of 25)

- [Wavelet Diffusion Neural Operator](https://mlsanity.com/paper/2412.04833/) (2024)
- [Effective control of two-dimensional Rayleigh--B'enard convection: invariant multi-agent reinforcement learning is all you need](https://mlsanity.com/paper/2304.02370/) (2023)
- [SINDy-RL: Interpretable and Efficient Model-Based Reinforcement Learning](https://mlsanity.com/paper/2403.09110/) (2024)
- [Physics-informed Dyna-Style Model-Based Deep Reinforcement Learning for Dynamic Control](https://mlsanity.com/paper/2108.00128/) (2021)
- [Data-driven control of spatiotemporal chaos with reduced-order neural ODE-based models and reinforcement learning](https://mlsanity.com/paper/2205.00579/) (2022)
- [Hybrid analysis and modeling, eclecticism, and multifidelity computing toward digital twin revolution](https://mlsanity.com/paper/2103.14629/) (2021)
- [Thermal Control of Laser Powder Bed Fusion Using Deep Reinforcement Learning](https://mlsanity.com/paper/2102.03355/) (2021)
- [Multi-fidelity reinforcement learning framework for shape optimization](https://mlsanity.com/paper/2202.11170/) (2022)
- [Multi-agent reinforcement learning for the control of three-dimensional Rayleigh-B'enard convection](https://mlsanity.com/paper/2407.21565/) (2024)
- [Parametric PDE Control with Deep Reinforcement Learning and Differentiable L0-Sparse Polynomial Policies](https://mlsanity.com/paper/2403.15267/) (2024)
- [Comparative analysis of machine learning methods for active flow control](https://mlsanity.com/paper/2202.11664/) (2022)
- [Koopman-Based Surrogate Modelling of Turbulent Rayleigh-B'enard Convection](https://mlsanity.com/paper/2405.06425/) (2024)
- [HypeRL: Hypernetwork-Based Reinforcement Learning for Control of Parametrized Dynamical Systems](https://mlsanity.com/paper/2501.04538/) (2025)
- [Parallel bootstrap-based on-policy deep reinforcement learning for continuous flow control applications](https://mlsanity.com/paper/2304.12330/) (2023)
- [Control of Rayleigh-B'enard Convection: Effectiveness of Reinforcement Learning in the Turbulent Regime](https://mlsanity.com/paper/2504.12000/) (2025)
- [Revisiting Heat Flux Analysis of Tungsten Monoblock Divertor on EAST using Physics-Informed Neural Network](https://mlsanity.com/paper/2508.03776/) (2025)
- [Asynchronous Parallel Reinforcement Learning for Optimizing Propulsive Performance in Fin Ray Control](https://mlsanity.com/paper/2401.11349/) (2024)
- [Optimal Parallelization Strategies for Active Flow Control in Deep Reinforcement Learning-Based Computational Fluid Dynamics](https://mlsanity.com/paper/2402.11515/) (2024)
- [Reward hacking in physical reinforcement learning revealed by turbulent drag reduction](https://mlsanity.com/paper/2606.06227/) (2026)
- [Heuristic Learning for Active Flow Control Using Coding Agents](https://mlsanity.com/paper/2607.11565/) (2026)
- [Genetically-inspired convective heat transfer enhancement in a turbulent boundary layer](https://mlsanity.com/paper/2304.12618/) (2023)
- [Timescale Separation Enables Deep Reinforcement Learning Control of Rotating Detonation Engine Mode Transitions](https://mlsanity.com/paper/2604.14398/) (2026)
- [A perspective on fluid mechanical environments for challenges in reinforcement learning](https://mlsanity.com/paper/2605.25011/) (2026)
- [Physics-enhanced reinforcement learning for real-time optimal control of dynamical systems](https://mlsanity.com/paper/2607.16177/) (2026)
- [Generalized Kalman filter based temporal difference reinforcement learning](https://mlsanity.com/paper/2607.20010/) (2026)

## Abstract (arXiv)

Thermal convection is ubiquitous in nature as well as in many industrial applications. The identification of effective control strategies to, e.g., suppress or enhance the convective heat exchange under fixed external thermal gradients is an outstanding fundamental and technological issue. In this work, we explore a novel approach, based on a state-of-the-art Reinforcement Learning (RL) algorithm, which is capable of significantly reducing the heat transport in a two-dimensional Rayleigh-B'enard system by applying small temperature fluctuations to the lower boundary of the system. By using numerical simulations, we show that our RL-based control is able to stabilize the conductive regime and bring the onset of convection up to a Rayleigh number Rac ≈ 3 · 104, whereas in the uncontrolled case it holds Rac=1708. Additionally, for Ra > 3 · 104, our approach outperforms other state-of-the-art control algorithms reducing the heat flux by a factor of about 2.5. In the last part of the manuscript, we address theoretical limits connected to controlling an unstable and chaotic dynamics as the one considered here. We show that controllability is hindered by observability and/or capabilities of actuating actions, which can be quantified in terms of characteristic time delays. When these delays become comparable with the Lyapunov time of the system, control becomes impossible.