mlsanityData to 7 Oct 2026

Offline Reinforcement Learning for Fluid Controls: Data-based Multi-observational Policy Extraction

arXiv 2606.31025 ↗First posted 30 Jun 2026cs.LGcited by 0

On the map: Offline RL and imitation learning from data → Offline RL algorithms

Abstract (arXiv)

Active flow control is a fundamental application in engineering. Recent advances in deep reinforcement learning have made progress in this field. However, the classical online RL approaches require extensive real-time interactions with the high fidelity environment, while each sensor configuration change necessitates whole policy retraining. All these factors result in prohibitive computational costs for real-world applications. In this work, we propose a novel offline RL framework that addresses both challenges through data-driven policy extraction. We develop a sensor position-conditioned architecture that enables a single policy network to adapt seamlessly to multiple sensor arrangements. The position-conditioned approach incorporated spatial relationship modeling through Point Attention layers to ensure the generalizability to varying sensor placements. We demonstrate the framework on two representative problems, mitigating chaoticity in the Kuramoto-Sivashinsky equation and flow control over airfoils governed by the Navier-Stokes equation. The result demonstrates that the policy extraction from the dataset provides unprecedented flexibility for sensor placement optimization. This approach represents a significant step towards adaptive, intelligent flow control systems.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Offline RL algorithms, one of 13 directions in Offline RL and imitation learning from data. Builds on 22 of its 57 references that are in the map. First posted on arXiv 30 Jun 2026.

Builds on

22 of 57 references.

Closest ancestors · 5

Same direction, newest first

  1. 2022A Survey on Offline Reinforcement Learning: Taxonomy, Review, and Open Problems2 Mar
  2. 2021Uncertainty-Based Offline Reinforcement Learning with Diversified Q-Ensemble4 Oct
  3. 2020Conservative Q-Learning for Offline Reinforcement Learning8 Jun
  4. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems4 May
  5. 2019Behavior Regularized Offline Reinforcement Learning26 Nov

Other references · 17

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  2. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  3. 2020Point Transformercited by 1,462
  4. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,227
  5. 2019Stabilizing Off-Policy Q-Learning via Bootstrapping Error Reductioncited by 833
  6. 2019End-to-End Robotic Reinforcement Learning without Reward Engineeringcited by 115
  7. 2020Learning to Generalize Across Long-Horizon Tasks from Human Demonstrationscited by 114
  8. 2018Efficient collective swimming by harnessing vortices through deep reinforcement learningcited by 73
  9. 2021Scientific multi-agent reinforcement learning for wall-models of turbulent flowscited by 52
  10. 2022Multi-resolution partial differential equations preserved learning framework for spatiotemporal dynamicscited by 44
  11. 2017Flow Navigation by Smart Microswimmers via Reinforcement Learningcited by 32
  12. 2023Effective control of two-dimensional Rayleigh--B'enard convection: invariant multi-agent reinforcement learning is all you needcited by 25
5 more
  1. 2021Physics-informed Dyna-Style Model-Based Deep Reinforcement Learning for Dynamic Controlcited by 22
  2. 2017Finding Efficient Swimming Strategies in a Three Dimensional Chaotic Flow by Reinforcement Learningcited by 11
  3. 2022Differentiable physics-enabled closure modeling for Burgers' turbulencecited by 7
  4. 2024Enhancing Vehicle Aerodynamics with Deep Reinforcement Learning in Voxelised Modelscited by 3
  5. 2025Multi-fidelity Reinforcement Learning Control for Complex Dynamical Systemscited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Heuristic Learning for Active Flow Control Using Coding Agents
  2. 2026Flow Actor-Critic for Offline Reinforcement Learning
  3. 2022Flow to Control: Offline Reinforcement Learning with Lossless Primitive Discovery
  4. 2026Reinforcement Learning for Flow-Matching Policies with Density Transport
  5. 2024FlowRL: Flow-Augmented Few-Shot Reinforcement Learning for Semi-Structured Sensor Data
  6. 2024Real-World Fluid Directed Rigid Body Control via Deep Reinforcement Learning
  7. 2022Regularizing a Model-based Policy Stationary Distribution to Stabilize Offline Reinforcement Learning
  8. 2026Towards Efficient and Expressive Offline RL via Flow-Anchored Noise-conditioned Q-Learning