# Differentiable Reinforcement Learning for Path Tracking by an Agile Fish-Like Robot

arXiv: https://arxiv.org/abs/2607.16508 · HTML: https://mlsanity.com/paper/2607.16508/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for robot navigation and UAV control, one of 15 directions in Deep reinforcement learning methods.
- Builds on 5 of its 27 references that are in the map.
- First posted on arXiv 17 Jul 2026.

## Placed in

- Deep reinforcement learning methods / [RL for robot navigation and UAV control](https://mlsanity.com/deeprl/rl-for-robot-navigation-and-uav-control/)

## Builds on (most cited first, 5 of 5)

- [Curriculum Learning for Reinforcement Learning Domains: A Framework and Survey](https://mlsanity.com/paper/2003.04960/) (2020)
- [Accelerated Policy Learning with Parallel Differentiable Simulation](https://mlsanity.com/paper/2204.07137/) (2022)
- [Learning Quadruped Locomotion Using Differentiable Simulation](https://mlsanity.com/paper/2403.14864/) (2024)
- [Derivative-Based Koopman Operators for Real-Time Control of Robotic Systems](https://mlsanity.com/paper/2010.05778/) (2020)
- [Scaling Rough Terrain Locomotion with Automatic Curriculum Reinforcement Learning](https://mlsanity.com/paper/2601.17428/) (2026)

## Abstract (arXiv)

Fish-like swimming has inspired the design of several dozens if not hundreds of bioinspired robots in the last few decades. But the control and motion planning of such robots has been challenging due to the poorly modeled fluid-structure interaction and the nonlinear underactuated dynamics of such robots. While reinforcement learning has allowed significant advances in the context of ground and aerial robots, the lack of a suitable simulation environment with appropriate computational speed and accuracy have prevented similar progress for fish-like robots. We address this two-fold problem by developing a simulation platform that approximates the motion of our fish-like robot with computational efficiency. Then the motion control and path tracking by the robot is performed using PID control where the (variable) gains are learned using back propagation through time and training on a curriculum. The policy learned in the simulation is then applied on the physical platform, demonstrating an excellent match.