# Multi-agent reinforcement learning for the control of three-dimensional Rayleigh-B'enard convection

arXiv: https://arxiv.org/abs/2407.21565 · HTML: https://mlsanity.com/paper/2407.21565/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 70 references that are in the map.
- First posted on arXiv 31 Jul 2024.

## Builds on (most cited first, 12 of 12)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [Machine Learning for Fluid Mechanics](https://mlsanity.com/paper/1905.11075/) (2019)
- [Efficient collective swimming by harnessing vortices through deep reinforcement learning](https://mlsanity.com/paper/1802.02674/) (2018)
- [A review on Deep Reinforcement Learning for Fluid Mechanics](https://mlsanity.com/paper/1908.04127/) (2019)
- [Scientific multi-agent reinforcement learning for wall-models of turbulent flows](https://mlsanity.com/paper/2106.11144/) (2021)
- [Control of chaotic systems by Deep Reinforcement Learning](https://mlsanity.com/paper/1906.07672/) (2019)
- [Effective control of two-dimensional Rayleigh--B'enard convection: invariant multi-agent reinforcement learning is all you need](https://mlsanity.com/paper/2304.02370/) (2023)
- [Controlling Rayleigh-B'enard convection via Reinforcement Learning](https://mlsanity.com/paper/2003.14358/) (2020)
- [Dynamic Feature-based Deep Reinforcement Learning for Flow Control of Circular Cylinder with Sparse Surface Pressure Sensing](https://mlsanity.com/paper/2307.01995/) (2023)
- [Comparative analysis of machine learning methods for active flow control](https://mlsanity.com/paper/2202.11664/) (2022)

## Built on by (most cited first, 8 of 8)

- [Real-time optimal control of high-dimensional parametrized systems by deep learning-based reduced order models](https://mlsanity.com/paper/2409.05709/) (2024)
- [Surrogate Modeling of 3D Rayleigh-Benard Convection with Equivariant Autoencoders](https://mlsanity.com/paper/2505.13569/) (2025)
- [Control of Rayleigh-B'enard Convection: Effectiveness of Reinforcement Learning in the Turbulent Regime](https://mlsanity.com/paper/2504.12000/) (2025)
- [TianJi:An autonomous AI meteorologist for discovering physical mechanisms in atmospheric science](https://mlsanity.com/paper/2603.27738/) (2026)
- [Latent feedback control of distributed systems in multiple scenarios through deep learning-based reduced order models](https://mlsanity.com/paper/2412.09942/) (2024)
- [Deep Reinforcement Learning in Action: Real-Time Control of Vortex-Induced Vibrations](https://mlsanity.com/paper/2509.24556/) (2025)
- [Koopman-based surrogate modeling for reinforcement-learning-control of Rayleigh-Benard convection](https://mlsanity.com/paper/2603.28074/) (2026)
- [Timescale Separation Enables Deep Reinforcement Learning Control of Rotating Detonation Engine Mode Transitions](https://mlsanity.com/paper/2604.14398/) (2026)

## Abstract (arXiv)

Deep reinforcement learning (DRL) has found application in numerous use-cases pertaining to flow control. Multi-agent RL (MARL), a variant of DRL, has shown to be more effective than single-agent RL in controlling flows exhibiting locality and translational invariance. We present, for the first time, an implementation of MARL-based control of three-dimensional Rayleigh-B'enard convection (RBC). Control is executed by modifying the temperature distribution along the bottom wall divided into multiple control segments, each of which acts as an independent agent. Two regimes of RBC are considered at Rayleigh numbers Ra=500 and 750. Evaluation of the learned control policy reveals a reduction in convection intensity by 23.5% and 8.7% at Ra=500 and 750, respectively. The MARL controller converts irregularly shaped convective patterns to regular straight rolls with lower convection that resemble flow in a relatively more stable regime. We draw comparisons with proportional control at both Ra and show that MARL is able to outperform the proportional controller. The learned control strategy is complex, featuring different non-linear segment-wise actuator delays and actuation magnitudes. We also perform successful evaluations on a larger domain than used for training, demonstrating that the invariant property of MARL allows direct transfer of the learnt policy.