# Zero-shot Whole-Body Manipulation with a Large-Scale Soft Robotic Torso via Guided Reinforcement Learning

arXiv: https://arxiv.org/abs/2509.23556 · HTML: https://mlsanity.com/paper/2509.23556/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods.
- Builds on 15 of its 49 references that are in the map.
- First posted on arXiv 28 Sep 2025.

## Placed in

- Deep reinforcement learning methods / [RL for robot locomotion and manipulation](https://mlsanity.com/deeprl/rl-for-robot-locomotion-and-manipulation/)

## Builds on (most cited first, 15 of 15)

- [Overcoming Exploration in Reinforcement Learning with Demonstrations](https://mlsanity.com/paper/1709.10089/) (2017)
- [How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learned](https://mlsanity.com/paper/2102.02915/) (2021)
- [A Survey of Imitation Learning: Algorithms, Recent Developments, and Challenges](https://mlsanity.com/paper/2309.02473/) (2023)
- [Time Limits in Reinforcement Learning](https://mlsanity.com/paper/1712.00378/) (2017)
- [Demonstration-Guided Reinforcement Learning with Learned Skills](https://mlsanity.com/paper/2107.10253/) (2021)
- [Model Based Control of Soft Robots: A Survey of the State of the Art and Open Challenges](https://mlsanity.com/paper/2110.01358/) (2021)
- [Soft Robots Modeling: a Structured Overview](https://mlsanity.com/paper/2112.03645/) (2021)
- [Elastica: A compliant mechanics environment for soft robotic control](https://mlsanity.com/paper/2009.08422/) (2020)
- [Jerk-limited Real-time Trajectory Generation with Arbitrary Target States](https://mlsanity.com/paper/2105.04830/) (2021)
- [Improved Learning of Robot Manipulation Tasks via Tactile Intrinsic Motivation](https://mlsanity.com/paper/2102.11051/) (2021)
- [Sim-to-Real for Soft Robots using Differentiable FEM: Recipes for Meshing, Damping, and Actuation](https://mlsanity.com/paper/2201.12560/) (2022)
- [Plan-Guided Reinforcement Learning for Whole-Body Manipulation](https://mlsanity.com/paper/2310.12263/) (2023)
- [Baloo: A Large-Scale Hybrid Soft Robotic Torso for Whole-Arm Manipulation](https://mlsanity.com/paper/2409.08420/) (2024)
- [Learning Dynamic Tasks on a Large-scale Soft Robot in a Handful of Trials](https://mlsanity.com/paper/2411.07342/) (2024)
- [PneuDrive: An Embedded Pressure Control System and Modeling Toolkit for Large-Scale Soft Robots](https://mlsanity.com/paper/2504.00222/) (2025)

## Abstract (arXiv)

Whole-body manipulation is a powerful yet underexplored approach that enables robots to interact with large, heavy, or awkward objects using more than just their end-effectors. Soft robots, with their inherent passive compliance, are particularly well-suited for such contact-rich manipulation tasks, but their uncertainties in kinematics and dynamics pose significant challenges for simulation and control. In this work, we address this challenge with a simulation that can run up to 350x real time on a single thread in MuJoCo and provide a detailed analysis of the critical tradeoffs between speed and accuracy for this simulation. Using this framework, we demonstrate a successful zero-shot sim-to-real transfer of a learned whole-body manipulation policy, achieving an 88% success rate on the Baloo hardware platform. We show that guiding RL with a simple motion primitive is critical to this success where standard reward shaping methods struggled to produce a stable and successful policy for whole-body manipulation. Furthermore, our analysis reveals that the learned policy does not simply mimic the motion primitive. It exhibits beneficial reactive behavior, such as re-grasping and perturbation recovery. We analyze and contrast this learned policy against an open-loop baseline to show that the policy can also exhibit aggressive over-corrections under perturbation. To our knowledge, this is the first demonstration of forceful, six-DoF whole-body manipulation using two continuum soft arms on a large-scale platform (10 kg payloads), with zero-shot policy transfer.