mlsanityData to 14 Oct 2026

Asynchronous Parallel Reinforcement Learning for Optimizing Propulsive Performance in Fin Ray Control

arXiv 2401.11349 ↗First posted 21 Jan 2024physics.flu-dyn cs.LG cs.SYcited by 1

On the map: Deep reinforcement learning methods → RL for robot navigation and UAV control

Abstract (arXiv)

Fish fin rays constitute a sophisticated control system for ray-finned fish, facilitating versatile locomotion within complex fluid environments. Despite extensive research on the kinematics and hydrodynamics of fish locomotion, the intricate control strategies in fin-ray actuation remain largely unexplored. While deep reinforcement learning (DRL) has demonstrated potential in managing complex nonlinear dynamics; its trial-and-error nature limits its application to problems involving computationally demanding environmental interactions. This study introduces a cutting-edge off-policy DRL algorithm, interacting with a fluid-structure interaction (FSI) environment to acquire intricate fin-ray control strategies tailored for various propulsive performance objectives. To enhance training efficiency and enable scalable parallelism, an innovative asynchronous parallel training (APT) strategy is proposed, which fully decouples FSI environment interactions and policy/value network optimization. The results demonstrated the success of the proposed method in discovering optimal complex policies for fin-ray actuation control, resulting in a superior propulsive performance compared to the optimal sinusoidal actuation function identified through a parametric grid search. The merit and effectiveness of the APT approach are also showcased through comprehensive comparison with conventional DRL training strategies in numerical experiments of controlling nonlinear dynamics.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in RL for robot navigation and UAV control, one of 15 directions in Deep reinforcement learning methods. Builds on 21 of its 55 references that are in the map. First posted on arXiv 21 Jan 2024.

Builds on

21 of 55 references.

References · 21

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  3. 2015Continuous control with deep reinforcement learningcited by 3,407
  4. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  5. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,258
  6. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,353
  7. 2015Prioritized Experience Replaycited by 1,099
  8. 2019Solving Rubik's Cube with a Robot Handcited by 734
  9. 2016Deep Reinforcement Learning for Robotic Manipulation with Asynchronous Off-Policy Updatescited by 353
  10. 2020Agent57: Outperforming the Atari Human Benchmarkcited by 303
  11. 2019Soft Actor-Critic for Discrete Action Settingscited by 174
  12. 2018Efficient collective swimming by harnessing vortices through deep reinforcement learningcited by 73
9 more
  1. 2019A review on Deep Reinforcement Learning for Fluid Mechanicscited by 66
  2. 2017Flow Navigation by Smart Microswimmers via Reinforcement Learningcited by 34
  3. 2019Control of chaotic systems by Deep Reinforcement Learningcited by 29
  4. 2020Controlling Rayleigh-B'enard convection via Reinforcement Learningcited by 25
  5. 2021Bayesian Sequential Optimal Experimental Design for Nonlinear Models Using Policy Gradient Reinforcement Learningcited by 24
  6. 2021Physics-informed Dyna-Style Model-Based Deep Reinforcement Learning for Dynamic Controlcited by 23
  7. 2017Adaptive Traffic Signal Control: Deep Reinforcement Learning Algorithm with Experience Replay and Target Networkcited by 16
  8. 2020Single-step deep reinforcement learning for open-loop control of laminar and turbulent flowscited by 14
  9. 2017Finding Efficient Swimming Strategies in a Three Dimensional Chaotic Flow by Reinforcement Learningcited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Knowledge-Constrained Shape Optimization with a Mixture-of-Experts Neural Operator for High-Confidence Designcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Enhancing Efficiency and Propulsion in Bio-mimetic Robotic Fish through End-to-End Deep Reinforcement Learning
  2. 2024Learning Agile Swimming: An End-to-End Approach without CPGs
  3. 2025Underwater Soft Fin Flapping Motion with Deep Neural Network Based Surrogate Model
  4. 2026Egocentric Station Holding of Robotic Fish in Unknown Turbulent Background Flow ↗
  5. 2024Efficient Navigation of a Robotic Fish Swimming Across the Vortical Flow Field
  6. 2026Generalizing deep reinforcement learning across cable-driven parallel robot configurations with actuator-level policies
  7. 2026SWIM: Compact Environment Representation for Reinforcement Learning Human Swimming
  8. 2024Optimal Parallelization Strategies for Active Flow Control in Deep Reinforcement Learning-Based Computational Fluid Dynamics