mlsanityData to 7 Oct 2026

Multi-agent reinforcement learning for the control of three-dimensional Rayleigh-B'enard convection

arXiv 2407.21565 ↗First posted 31 Jul 2024physics.flu-dyn cs.LGcited by 8

Not on the map. Why?

Abstract (arXiv)

Deep reinforcement learning (DRL) has found application in numerous use-cases pertaining to flow control. Multi-agent RL (MARL), a variant of DRL, has shown to be more effective than single-agent RL in controlling flows exhibiting locality and translational invariance. We present, for the first time, an implementation of MARL-based control of three-dimensional Rayleigh-B'enard convection (RBC). Control is executed by modifying the temperature distribution along the bottom wall divided into multiple control segments, each of which acts as an independent agent. Two regimes of RBC are considered at Rayleigh numbers Ra=500 and 750. Evaluation of the learned control policy reveals a reduction in convection intensity by 23.5% and 8.7% at Ra=500 and 750, respectively. The MARL controller converts irregularly shaped convective patterns to regular straight rolls with lower convection that resemble flow in a relatively more stable regime. We draw comparisons with proportional control at both Ra and show that MARL is able to outperform the proportional controller. The learned control strategy is complex, featuring different non-linear segment-wise actuator delays and actuation magnitudes. We also perform successful evaluations on a larger domain than used for training, demonstrating that the invariant property of MARL allows direct transfer of the learnt policy.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 12 of its 70 references that are in the map. First posted on arXiv 31 Jul 2024.

Builds on

12 of 70 references.

References · 12

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  3. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,704
  4. 2019Machine Learning for Fluid Mechanicscited by 380
  5. 2018Efficient collective swimming by harnessing vortices through deep reinforcement learningcited by 73
  6. 2019A review on Deep Reinforcement Learning for Fluid Mechanicscited by 65
  7. 2021Scientific multi-agent reinforcement learning for wall-models of turbulent flowscited by 52
  8. 2019Control of chaotic systems by Deep Reinforcement Learningcited by 28
  9. 2020Controlling Rayleigh-B'enard convection via Reinforcement Learningcited by 25
  10. 2023Effective control of two-dimensional Rayleigh--B'enard convection: invariant multi-agent reinforcement learning is all you needcited by 25
  11. 2023Dynamic Feature-based Deep Reinforcement Learning for Flow Control of Circular Cylinder with Sparse Surface Pressure Sensingcited by 7
  12. 2022Comparative analysis of machine learning methods for active flow controlcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 3; 2026: 3.

8 papers.

  1. 2024Real-time optimal control of high-dimensional parametrized systems by deep learning-based reduced order modelscited by 4
  2. 2025Surrogate Modeling of 3D Rayleigh-Benard Convection with Equivariant Autoencoderscited by 3
  3. 2025Control of Rayleigh-B'enard Convection: Effectiveness of Reinforcement Learning in the Turbulent Regimecited by 2
  4. 2026TianJi:An autonomous AI meteorologist for discovering physical mechanisms in atmospheric sciencecited by 2
  5. 2024Latent feedback control of distributed systems in multiple scenarios through deep learning-based reduced order modelscited by 0
  6. 2025Deep Reinforcement Learning in Action: Real-Time Control of Vortex-Induced Vibrationscited by 0
  7. 2026Koopman-based surrogate modeling for reinforcement-learning-control of Rayleigh-Benard convectioncited by 0
  8. 2026Timescale Separation Enables Deep Reinforcement Learning Control of Rotating Detonation Engine Mode Transitionscited by 0