# Optimizing Metachronal Paddling with Reinforcement Learning at Low Reynolds Number

arXiv: https://arxiv.org/abs/2507.18849 · HTML: https://mlsanity.com/paper/2507.18849/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for robot navigation and UAV control, one of 15 directions in Deep reinforcement learning methods.
- Builds on 9 of its 33 references that are in the map.
- First posted on arXiv 24 Jul 2025.

## Placed in

- Deep reinforcement learning methods / [RL for robot navigation and UAV control](https://mlsanity.com/deeprl/rl-for-robot-navigation-and-uav-control/)

## Builds on (most cited first, 9 of 9)

- [Deep Reinforcement Learning: An Overview](https://mlsanity.com/paper/1701.07274/) (2017)
- [Efficient collective swimming by harnessing vortices through deep reinforcement learning](https://mlsanity.com/paper/1802.02674/) (2018)
- [A review on Deep Reinforcement Learning for Fluid Mechanics](https://mlsanity.com/paper/1908.04127/) (2019)
- [On the Convergence and Sample Complexity Analysis of Deep Q-Networks with ε-Greedy Exploration](https://mlsanity.com/paper/2310.16173/) (2023)
- [Learning to swim in potential flow](https://mlsanity.com/paper/2009.14280/) (2020)
- [Learning swimming via deep reinforcement learning](https://mlsanity.com/paper/2209.10935/) (2022)
- [Deep Dive into Model-free Reinforcement Learning for Biological and Robotic Systems: Theory and Practice](https://mlsanity.com/paper/2405.11457/) (2024)
- [Deep reinforcement learning for tracking a moving target in jellyfish-like swimming](https://mlsanity.com/paper/2409.08815/) (2024)
- [Navigation of a Three-Link Microswimmer via Deep Reinforcement Learning](https://mlsanity.com/paper/2506.00084/) (2025)

## Abstract (arXiv)

Metachronal paddling is a swimming strategy in which an organism oscillates sets of adjacent limbs with a constant phase lag, propagating a metachronal wave through its limbs and propelling it forward. This limb coordination strategy is utilized by swimmers across a wide range of Reynolds numbers, which suggests that this metachronal rhythm was selected for its optimality of swimming performance. In this study, we apply reinforcement learning to a swimmer at zero Reynolds number and investigate whether the learning algorithm selects this metachronal rhythm, or if other coordination patterns emerge. We design the swimmer agent with an elongated body and pairs of straight, inflexible paddles placed along the body for various fixed paddle spacings. Based on paddle spacing, the swimmer agent learns qualitatively different coordination patterns. At tight spacings, a back-to-front metachronal wave-like stroke emerges which resembles the commonly observed biological rhythm, but at wide spacings, different limb coordinations are selected. Across all resulting strokes, the fastest stroke is dependent on the number of paddles, however, the most efficient stroke is a back-to-front wave-like stroke regardless of the number of paddles.