mlsanityData to 7 Oct 2026

Advanced deep-reinforcement-learning methods for flow control: group-invariant and positional-encoding networks improve learning speed and quality

arXiv 2407.17822 ↗First posted 25 Jul 2024cs.LG physics.flu-dyncited by 5

Not on the map. Why?

Abstract (arXiv)

Flow control is key to maximize energy efficiency in a wide range of applications. However, traditional flow-control methods face significant challenges in addressing non-linear systems and high-dimensional data, limiting their application in realistic energy systems. This study advances deep-reinforcement-learning (DRL) methods for flow control, particularly focusing on integrating group-invariant networks and positional encoding into DRL architectures. Our methods leverage multi-agent reinforcement learning (MARL) to exploit policy invariance in space, in combination with group-invariant networks to ensure local symmetry invariance. Additionally, a positional encoding inspired by the transformer architecture is incorporated to provide location information to the agents, mitigating action constraints from strict invariance. The proposed methods are verified using a case study of Rayleigh-B'enard convection, where the goal is to minimize the Nusselt number Nu. The group-invariant neural networks (GI-NNs) show faster convergence compared to the base MARL, achieving better average policy performance. The GI-NNs not only cut DRL training time in half but also notably enhance learning reproducibility. Positional encoding further enhances these results, effectively reducing the minimum Nu and stabilizing convergence. Interestingly, group invariant networks specialize in improving learning speed and positional encoding specializes in improving learning quality. These results demonstrate that choosing a suitable feature-representation method according to the purpose as well as the characteristics of each control problem is essential. We believe that the results of this study will not only inspire novel DRL methods with invariant and unique representations, but also provide useful insights for industrial applications.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 22 of its 49 references that are in the map. First posted on arXiv 25 Jul 2024.

Builds on

22 of 49 references.

References · 22

  1. 2017Attention Is All You Needcited by 59,932
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2015Trust Region Policy Optimizationcited by 2,503
  4. 2016Group Equivariant Convolutional Networkscited by 1,154
  5. 2020Learning Invariant Representations for Reinforcement Learning without Reconstructioncited by 424
  6. 2020A Survey of Convolutional Neural Networks: Analysis, Applications, and Prospectscited by 396
  7. 2022Exploration in Deep Reinforcement Learning: A Surveycited by 158
  8. 2021Recent Advances in Reinforcement Learning in Financecited by 102
  9. 2015Rotation-invariant convolutional neural networks for galaxy morphology predictioncited by 83
  10. 2022SO(2)-Equivariant Reinforcement Learningcited by 79
  11. 2019A review on Deep Reinforcement Learning for Fluid Mechanicscited by 65
  12. 2020Roto-Translation Equivariant Convolutional Networks: Application to Histopathology Image Analysiscited by 32
10 more
  1. 2023Effective control of two-dimensional Rayleigh--B'enard convection: invariant multi-agent reinforcement learning is all you needcited by 25
  2. 2024SINDy-RL: Interpretable and Efficient Model-Based Reinforcement Learningcited by 24
  3. 2023Distributed Control of Partial Differential Equations Using Convolutional Reinforcement Learningcited by 21
  4. 2022Towards Applicable Reinforcement Learning: Improving the Generalization and Sample Efficiency with Policy Ensemblecited by 18
  5. 2022Deep Reinforcement Learning for Turbulence Modeling in Large Eddy Simulationscited by 16
  6. 2022Residual-based physics-informed transfer learning: A hybrid method for accelerating long-term CFD simulations via deep learningcited by 14
  7. 2021Finite volume method network for acceleration of unsteady computational fluid dynamics: non-reacting and reacting flowscited by 12
  8. 2021Symmetry reduction for deep reinforcement learning active control of chaotic spatiotemporal dynamicscited by 10
  9. 2018LIFT: Reinforcement Learning in Computer Systems by Learning From Demonstrationscited by 8
  10. 2023Active flow control for three-dimensional cylinders through deep reinforcement learningcited by 3

Built on by

Cited by year

Citing papers per year: 2024: 2; 2025: 3.

5 papers.

  1. 2024Real-time optimal control of high-dimensional parametrized systems by deep learning-based reduced order modelscited by 4
  2. 2025Surrogate Modeling of 3D Rayleigh-Benard Convection with Equivariant Autoencoderscited by 3
  3. 2025Control of Rayleigh-B'enard Convection: Effectiveness of Reinforcement Learning in the Turbulent Regimecited by 2
  4. 2025Invariant Control Strategies for Active Flow Control using Graph Neural Networkscited by 1
  5. 2024Latent feedback control of distributed systems in multiple scenarios through deep learning-based reduced order modelscited by 0