Flow-aware Optimal Navigation in Unsteady Flows through Reinforcement Learning
On the map: Deep reinforcement learning methods → RL for robot navigation and UAV control
Abstract (arXiv)
Autonomous robotic navigation in nonstationary time-varying fluid flows remains a fundamental challenge due to partial observability and the unpredictability of realistic environments. While classical optimal control frameworks employed in robotics require unrealistic a-priori global flow knowledge, biological systems are able to navigate successfully by exploiting localized sensory cues. In this work we present a reinforcement learning approach using the TD3 algorithm to train autonomous agents to reach arbitrary targets within a parametric, chaotic double-gyre flow. To investigate optimal sensory mechanisms, we evaluate five bio-inspired observation strategies based on relative position, local velocity or local vorticity measures, and short-term memory variants. Additionally, we analyze the impact of providing agents with explicit global flow parameters. Numerical results demonstrate that an agent that is able to sense and remember a set number of flow velocity measures achieves the highest performance. The experiments reveal a trade-off in sensor utility: velocity-aware agents optimize energy efficiency, whereas vorticity sensors provide superior structural mapping and achieve better target proximity. Incorporating explicit global flow parameters is shown to decrease navigation performance. This behavior suggests that reinforcement learning-based autonomous systems develop more robust and general policies when restricted to implicit flow representations. The presented results offer insights for improving the transition of bio-inspired robotic navigation from simulation to real-world environments.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for robot navigation and UAV control, one of 15 directions in Deep reinforcement learning methods. Builds on 8 of its 18 references that are in the map. First posted on arXiv 15 Jul 2026.
Builds on
8 of 18 references.
Closest ancestors · 1
Same direction, newest first
- 2024Reinforcement learning16 May
Other references · 7
- 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
- 2017Flow Navigation by Smart Microswimmers via Reinforcement Learningcited by 34
- 2021Learning Efficient Navigation in Vortical Flow Fieldscited by 26
- 2017Finding Efficient Swimming Strategies in a Three Dimensional Chaotic Flow by Reinforcement Learningcited by 11
- 2023Finite Time Lyapunov Exponent Analysis of Model Predictive Control and Reinforcement Learningcited by 7
- 2021Finite-Horizon, Energy-Optimal Trajectories in Unsteady Flowscited by 3
- 2025HypeRL: Hypernetwork-Based Reinforcement Learning for Control of Parametrized Dynamical Systemscited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025A critical assessment of reinforcement learning methods for microswimmer navigation in complex flows
- 2025Khalasi: Energy-Efficient Navigation for Surface Vehicles in Vortical Flow Fields
- 2024Fish-inspired tracking of underwater turbulent plumes
- 2024Efficient Navigation of a Robotic Fish Swimming Across the Vortical Flow Field
- 2026Emergence of a Flow-Assisted Casting Strategy for Olfactory Navigation via Memory-Augmented Reinforcement Learning
- 2026Egocentric Station Holding of Robotic Fish in Unknown Turbulent Background Flow ↗
- 2026Self-Evolving Scientific Agent Designs Physically Reasoned White-Box Fluid Control
- 2025Navigation of a Three-Link Microswimmer via Deep Reinforcement Learning