mlsanityData to 14 Oct 2026

Effective control of two-dimensional Rayleigh--B'enard convection: invariant multi-agent reinforcement learning is all you need

arXiv 2304.02370 ↗First posted 5 Apr 2023physics.flu-dyn cs.LGcited by 27

Not on the map. Why?

Abstract (arXiv)

Rayleigh-B'enard convection (RBC) is a recurrent phenomenon in several industrial and geoscience flows and a well-studied system from a fundamental fluid-mechanics viewpoint. However, controlling RBC, for example by modulating the spatial distribution of the bottom-plate heating in the canonical RBC configuration, remains a challenging topic for classical control-theory methods. In the present work, we apply deep reinforcement learning (DRL) for controlling RBC. We show that effective RBC control can be obtained by leveraging invariant multi-agent reinforcement learning (MARL), which takes advantage of the locality and translational invariance inherent to RBC flows inside wide channels. The MARL framework applied to RBC allows for an increase in the number of control segments without encountering the curse of dimensionality that would result from a naive increase in the DRL action-size dimension. This is made possible by the MARL ability for re-using the knowledge generated in different parts of the RBC domain. We show in a case study that MARL DRL is able to discover an advanced control strategy that destabilizes the spontaneous RBC double-cell pattern, changes the topology of RBC by coalescing adjacent convection cells, and actively controls the resulting coalesced cell to bring it to a new stable configuration. This modified flow configuration results in reduced convective heat transfer, which is beneficial in several industrial processes. Therefore, our work both shows the potential of MARL DRL for controlling large RBC systems, as well as demonstrates the possibility for DRL to discover strategies that move the RBC configuration between different topological configurations, yielding desirable heat-transfer characteristics. These results are useful for both gaining further understanding of the intrinsic properties of RBC, as well as for developing industrial applications.

Key facts. Cited by 27 papers in the map (data through 14 Oct 2026). Builds on 10 of its 90 references that are in the map. First posted on arXiv 5 Apr 2023.

Builds on

10 of 90 references.

References · 10

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  3. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,745
  4. 2017Deep Reinforcement Learning: An Overviewcited by 333
  5. 2021Tianshou: a Highly Modularized Deep Reinforcement Learning Librarycited by 143
  6. 2019A review on Deep Reinforcement Learning for Fluid Mechanicscited by 66
  7. 2019Control of chaotic systems by Deep Reinforcement Learningcited by 29
  8. 2020Controlling Rayleigh-B'enard convection via Reinforcement Learningcited by 25
  9. 2022Deep Reinforcement Learning for Computational Fluid Dynamics on HPC Systemscited by 7
  10. 2022Comparative analysis of machine learning methods for active flow controlcited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 3; 2024: 9; 2025: 6; 2026: 9.

27 papers.

Sort:

By area:

  1. 2024Multi-agent reinforcement learning for the control of three-dimensional Rayleigh-B'enard convectioncited by 8
  2. 2023Equivariance and partial observations in Koopman operator theory for partial differential equationscited by 6
  3. 2024Advanced deep-reinforcement-learning methods for flow control: group-invariant and positional-encoding networks improve learning speed and qualitycited by 5
  4. 2024Navigation in a simplified Urban Flow through Deep Reinforcement Learningcited by 5
  5. 2024Model-based deep reinforcement learning for accelerated learning from flow simulationscited by 4
  6. 2024Real-time optimal control of high-dimensional parametrized systems by deep learning-based reduced order modelscited by 4
  7. 2023Active flow control for three-dimensional cylinders through deep reinforcement learningcited by 3
  8. 2024Koopman-Based Surrogate Modelling of Turbulent Rayleigh-B'enard Convectioncited by 3
  9. 2025Surrogate Modeling of 3D Rayleigh-Benard Convection with Equivariant Autoencoderscited by 3
  10. 2023Gym-preCICE: Reinforcement Learning Environments for Active Flow Controlcited by 2
  11. 2024Solving Partial Differential Equations with Equivariant Extreme Learning Machinescited by 2
  12. 2025Multi-fidelity Reinforcement Learning Control for Complex Dynamical Systemscited by 2
15 more
  1. 2025Control of Rayleigh-B'enard Convection: Effectiveness of Reinforcement Learning in the Turbulent Regimecited by 2
  2. 2026Plug-and-Play Benchmarking of Reinforcement Learning Algorithms for Large-Scale Flow Controlcited by 2
  3. 2024Reinforcement Learning for Freeway Lane-Change Regulation via Connected Vehiclescited by 1
  4. 2025Understanding the role of autoencoders for stiff dynamical systems using information theorycited by 1
  5. 2025Invariant Control Strategies for Active Flow Control using Graph Neural Networkscited by 1
  6. 2025HypeMARL: Multi-Agent Reinforcement Learning For High-Dimensional, Parametric, and Distributed Systemscited by 1
  7. 2026Reward hacking in physical reinforcement learning revealed by turbulent drag reductioncited by 1
  8. 2024Latent feedback control of distributed systems in multiple scenarios through deep learning-based reduced order modelscited by 0
  9. 2026Koopman-based surrogate modeling for reinforcement-learning-control of Rayleigh-Benard convectioncited by 0
  10. 2026Hyperfastrl: Hypernetwork-based reinforcement learning for unified control of parametric chaotic PDEscited by 0
  11. 2026Timescale Separation Enables Deep Reinforcement Learning Control of Rotating Detonation Engine Mode Transitionscited by 0
  12. 2026Explainable deep reinforcement learning reveals energy-efficient control strategies for turbulent drag reductioncited by 0
  13. 2026Offline Reinforcement Learning for Fluid Controls: Data-based Multi-observational Policy Extractioncited by 0
  14. 2026Gradient-free learning of a closed-loop wall controller for turbulent drag reductioncited by 0
  15. 2026Physics-enhanced reinforcement learning for real-time optimal control of dynamical systemscited by 0