Timescale Separation Enables Deep Reinforcement Learning Control of Rotating Detonation Engine Mode Transitions
On the map: Deep reinforcement learning methods → RL theory and sample complexity
Abstract (arXiv)
Rotating detonation engines (RDEs) are a promising propulsion concept that may offer higher thermodynamic efficiency and specific impulse than conventional systems, but nonlinear phenomena, including transitions to oscillatory or chaotic propagation modes, can hinder practical operation. Deep Reinforcement Learning (DRL) has emerged as a promising method for controlling complex nonlinear dynamics such as those observed in RDEs. However, the multi-timescale nature of the RDE system makes direct application of DRL challenging. We address this challenge by reformulating the DRL problem in a moving reference frame that follows the detonation-wave pattern, making the wave structure appear quasi-steady to the agent. This reformulation enables scale separation between fast detonation propagation and slower operating-mode dynamics. We train DRL controllers to modulate spatially segmented injection pressure in a one-dimensional reduced-order RDE model and induce rapid transitions between different mode-locked states. Across a range of actuation periods, initial states, and target modes, controllers trained in the moving frame learn more reliably than those trained in a stationary frame and remain effective over a broader range of actuation periods. These results suggest that symmetry-aware moving reference frame formulations may be useful for related multiscale flow-control problems and that scale separation should be exploited whenever possible to enable DRL control of multi-timescale systems.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods. Builds on 13 of its 76 references that are in the map. First posted on arXiv 15 Apr 2026.
Builds on
13 of 76 references.
References · 13
- 2019A review on Deep Reinforcement Learning for Fluid Mechanicscited by 66
- 2023A Survey of Temporal Credit Assignment in Deep Reinforcement Learningcited by 59
- 2023Bridging RL Theory and Practice with the Effective Horizoncited by 31
- 2020Control Frequency Adaptation via Action Persistence in Batch Reinforcement Learningcited by 28
- 2023Effective control of two-dimensional Rayleigh--B'enard convection: invariant multi-agent reinforcement learning is all you needcited by 27
- 2020Controlling Rayleigh-B'enard convection via Reinforcement Learningcited by 25
- 2022Dynamic Decision Frequency with Continuous Optionscited by 13
- 2024Multi-agent reinforcement learning for the control of three-dimensional Rayleigh-B'enard convectioncited by 8
- 2023Dynamic Feature-based Deep Reinforcement Learning for Flow Control of Circular Cylinder with Sparse Surface Pressure Sensingcited by 7
- 2024Navigation in a simplified Urban Flow through Deep Reinforcement Learningcited by 5
- 2020A Reinforcement Learning Approach for Transient Control of Liquid Rocket Enginescited by 4
- 2020Data-driven Modeling of Rotating Detonation Wavescited by 3
1 more
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Deep Reinforcement Learning in Action: Real-Time Control of Vortex-Induced Vibrations
- 2024Interpretable and Efficient Data-driven Discovery and Control of Distributed Systems
- 2024SINDy-RL: Interpretable and Efficient Model-Based Reinforcement Learning
- 2024Real-World Fluid Directed Rigid Body Control via Deep Reinforcement Learning
- 2026Hypersonic Flow Control: Generalized Deep Reinforcement Learning for Hypersonic Intake Unstart Control under Uncertainty
- 2025Dyna-Style Reinforcement Learning Modeling and Control of Non-linear Dynamics
- 2025Nuclear Microreactor Control with Deep Reinforcement Learning
- 2025Multi-fidelity Reinforcement Learning Control for Complex Dynamical Systems