# Offline Goal-Conditioned Reinforcement Learning for Shape Control of Deformable Linear Objects

arXiv: https://arxiv.org/abs/2403.10290 · HTML: https://mlsanity.com/paper/2403.10290/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Robot manipulation imitation learning, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 22 of its 41 references that are in the map.
- First posted on arXiv 15 Mar 2024.

## Placed in

- Offline RL and imitation learning from data / [Robot manipulation imitation learning](https://mlsanity.com/offlinerl/robot-manipulation-imitation-learning/)

## Builds on (most cited first, 22 of 22)

- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems](https://mlsanity.com/paper/2005.01643/) (2020)
- [A Minimalist Approach to Offline Reinforcement Learning](https://mlsanity.com/paper/2106.06860/) (2021)
- [Deep Reinforcement Learning that Matters](https://mlsanity.com/paper/1709.06560/) (2017)
- [Multi-Goal Reinforcement Learning: Challenging Robotics Environments and Request for Research](https://mlsanity.com/paper/1802.09464/) (2018)
- [Goal-Conditioned Reinforcement Learning: Problems and Solutions](https://mlsanity.com/paper/2201.08299/) (2022)
- [Sim-to-Real Reinforcement Learning for Deformable Object Manipulation](https://mlsanity.com/paper/1806.07851/) (2018)
- [d3rlpy: An Offline Deep Reinforcement Learning Library](https://mlsanity.com/paper/2111.03788/) (2021)
- [Learning to Manipulate Deformable Objects without Demonstrations](https://mlsanity.com/paper/1910.13439/) (2019)
- [A Workflow for Offline Model-Free Robotic Reinforcement Learning](https://mlsanity.com/paper/2109.10813/) (2021)
- [Learning Rope Manipulation Policies Using Dense Object Descriptors Trained on Synthetic Depth Data](https://mlsanity.com/paper/2003.01835/) (2020)
- [Self-Supervised Learning of State Estimation for Manipulating Deformable Linear Objects](https://mlsanity.com/paper/1911.06283/) (2019)
- [Challenges and Outlook in Robotic Manipulation of Deformable Objects](https://mlsanity.com/paper/2105.01767/) (2021)
- [D2RL: Deep Dense Architectures in Reinforcement Learning](https://mlsanity.com/paper/2010.09163/) (2020)
- [What is Essential for Unseen Goal Generalization of Offline Goal-conditioned RL?](https://mlsanity.com/paper/2305.18882/) (2023)
- [Dynamic Cloth Manipulation with Deep Reinforcement Learning](https://mlsanity.com/paper/1910.14475/) (2019)
- [Global Model Learning for Large Deformation Control of Elastic Deformable Linear Objects: An Efficient and Adaptive Approach](https://mlsanity.com/paper/2205.04004/) (2022)
- [Lattice-based shape tracking and servoing of elastic objects](https://mlsanity.com/paper/2209.01832/) (2022)
- [GOATS: Goal Sampling Adaptation for Scooping with Curriculum Reinforcement Learning](https://mlsanity.com/paper/2303.05193/) (2023)
- [Multi Actor-Critic DDPG for Robot Action Space Decomposition: A Framework to Control Large 3D Deformation of Soft Linear Objects](https://mlsanity.com/paper/2312.04308/) (2023)
- [DexDLO: Learning Goal-Conditioned Dexterous Policy for Dynamic Manipulation of Deformable Linear Objects](https://mlsanity.com/paper/2312.15204/) (2023)
- [Learning Shape Control of Elastoplastic Deformable Linear Objects](https://mlsanity.com/paper/2208.02067/) (2022)

## Built on by (most cited first, 2 of 2)

- [DiSA-IQL: Offline Reinforcement Learning for Robust Soft Robot Control under Distribution Shifts](https://mlsanity.com/paper/2510.00358/) (2025)
- [Self-Curriculum Model-based Reinforcement Learning for Shape Control of Deformable Linear Objects](https://mlsanity.com/paper/2602.21816/) (2026)

## Abstract (arXiv)

Deformable objects present several challenges to the field of robotic manipulation. One of the tasks that best encapsulates the difficulties arising due to non-rigid behavior is shape control, which requires driving an object to a desired shape. While shape-servoing methods have been shown successful in contexts with approximately linear behavior, they can fail in tasks with more complex dynamics. We investigate an alternative approach, using offline RL to solve a planar shape control problem of a Deformable Linear Object (DLO). To evaluate the effect of material properties, two DLOs are tested namely a soft rope and an elastic cord. We frame this task as a goal-conditioned offline RL problem, and aim to learn to generalize to unseen goal shapes. Data collection and augmentation procedures are proposed to limit the amount of experimental data which needs to be collected with the real robot. We evaluate the amount of augmentation needed to achieve the best results, and test the effect of regularization through behavior cloning on the TD3+BC algorithm. Finally, we show that the proposed approach is able to outperform a shape-servoing baseline in a curvature inversion experiment.