# Real-Time Reinforcement Learning for Dynamic Tasks with a Parallel Soft Robot

arXiv: https://arxiv.org/abs/2509.19525 · HTML: https://mlsanity.com/paper/2509.19525/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods.
- Builds on 22 of its 53 references that are in the map.
- First posted on arXiv 23 Sep 2025.

## Placed in

- Deep reinforcement learning methods / [RL for robot locomotion and manipulation](https://mlsanity.com/deeprl/rl-for-robot-locomotion-and-manipulation/)

## Builds on (most cited first, 22 of 22)

- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [DayDreamer: World Models for Physical Robot Learning](https://mlsanity.com/paper/2206.14176/) (2022)
- [Curriculum Learning for Reinforcement Learning Domains: A Framework and Survey](https://mlsanity.com/paper/2003.04960/) (2020)
- [Challenges of Real-World Reinforcement Learning](https://mlsanity.com/paper/1904.12901/) (2019)
- [How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learned](https://mlsanity.com/paper/2102.02915/) (2021)
- [Reset-Free Reinforcement Learning via Multi-Task Learning: Learning Dexterous Manipulation Behaviors without Human Intervention](https://mlsanity.com/paper/2104.11203/) (2021)
- [Leave no Trace: Learning to Reset for Safe and Autonomous Reinforcement Learning](https://mlsanity.com/paper/1711.06782/) (2017)
- [Smooth Exploration for Robotic Reinforcement Learning](https://mlsanity.com/paper/2005.05719/) (2020)
- [Self-Improving Robots: End-to-End Autonomous Visuomotor Reinforcement Learning](https://mlsanity.com/paper/2303.01488/) (2023)
- [Don't Start From Scratch: Leveraging Prior Data to Automate Robotic Reinforcement Learning](https://mlsanity.com/paper/2207.04703/) (2022)
- [You Only Live Once: Single-Life Reinforcement Learning](https://mlsanity.com/paper/2210.08863/) (2022)
- [Grow Your Limits: Continuous Improvement with Real-World RL for Robotic Locomotion](https://mlsanity.com/paper/2310.17634/) (2023)
- [Modeling and Control of Soft Robots Using the Koopman Operator and Model Predictive Control](https://mlsanity.com/paper/1902.02827/) (2019)
- [Continual Reinforcement Learning in 3D Non-stationary Environments](https://mlsanity.com/paper/1905.10112/) (2019)
- [Model Based Control of Soft Robots: A Survey of the State of the Art and Open Challenges](https://mlsanity.com/paper/2110.01358/) (2021)
- [Single Episode Policy Transfer in Reinforcement Learning](https://mlsanity.com/paper/1910.07719/) (2019)
- [Automating Reinforcement Learning with Example-based Resets](https://mlsanity.com/paper/2204.02041/) (2022)
- [Maximum diffusion reinforcement learning](https://mlsanity.com/paper/2309.15293/) (2023)
- [One-Shot Reinforcement Learning for Robot Navigation with Interactive Replay](https://mlsanity.com/paper/1711.10137/) (2017)
- [Reset-Free Guided Policy Search: Efficient Deep Reinforcement Learning with Stochastic Initial States](https://mlsanity.com/paper/1610.01112/) (2016)
- [Dynamic Task Space Control Enables Soft Manipulators to Perform Real-World Tasks](https://mlsanity.com/paper/2201.02151/) (2022)
- [Automated Gait Generation For Walking, Soft Robotic Quadrupeds](https://mlsanity.com/paper/2310.00498/) (2023)

## Built on by (most cited first, 1 of 1)

- [Damage Adaptation in Seconds for Architected Materials](https://mlsanity.com/paper/2606.17394/) (2026)

## Abstract (arXiv)

Closed-loop control remains an open challenge in soft robotics. The nonlinear responses of soft actuators under dynamic loading conditions limit the use of analytic models for soft robot control. Traditional methods of controlling soft robots underutilize their configuration spaces to avoid nonlinearity, hysteresis, large deformations, and the risk of actuator damage. Furthermore, episodic data-driven control approaches such as reinforcement learning (RL) are traditionally limited by sample efficiency and inconsistency across initializations. In this work, we demonstrate RL for reliably learning control policies for dynamic balancing tasks in real-time single-shot hardware deployments. We use a deformable Stewart platform constructed using parallel, 3D-printed soft actuators based on motorized handed shearing auxetic (HSA) structures. By introducing a curriculum learning approach based on expanding neighborhoods of a known equilibrium, we achieve reliable single-deployment balancing at arbitrary coordinates. In addition to benchmarking the performance of model-based and model-free methods, we demonstrate that in a single deployment, Maximum Diffusion RL is capable of learning dynamic balancing after half of the actuators are effectively disabled, by inducing buckling and by breaking actuators with bolt cutters. Training occurs with no prior data, in as fast as 15 minutes, with performance nearly identical to the fully-intact platform. Single-shot learning on hardware facilitates soft robotic systems reliably learning in the real world and will enable more diverse and capable soft robots.