# Robot Deformable Object Manipulation via NMPC-generated Demonstrations in Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/2502.11375 · HTML: https://mlsanity.com/paper/2502.11375/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods.
- Builds on 13 of its 55 references that are in the map.
- First posted on arXiv 17 Feb 2025.

## Placed in

- Deep reinforcement learning methods / [RL for robot locomotion and manipulation](https://mlsanity.com/deeprl/rl-for-robot-locomotion-and-manipulation/)

## Builds on (most cited first, 13 of 13)

- [RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control](https://mlsanity.com/paper/2307.15818/) (2023)
- [OpenVLA: An Open-Source Vision-Language-Action Model](https://mlsanity.com/paper/2406.09246/) (2024)
- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [TinyVLA: Towards Fast, Data-Efficient Vision-Language-Action Models for Robotic Manipulation](https://mlsanity.com/paper/2409.12514/) (2024)
- [Sim-to-Real Reinforcement Learning for Deformable Object Manipulation](https://mlsanity.com/paper/1806.07851/) (2018)
- [Learning to Manipulate Deformable Objects without Demonstrations](https://mlsanity.com/paper/1910.13439/) (2019)
- [SpeedFolding: Learning Efficient Bimanual Folding of Garments](https://mlsanity.com/paper/2208.10552/) (2022)
- [Challenges and Outlook in Robotic Manipulation of Deformable Objects](https://mlsanity.com/paper/2105.01767/) (2021)
- [Dynamic Cloth Manipulation with Deep Reinforcement Learning](https://mlsanity.com/paper/1910.14475/) (2019)
- [Folding Deformable Objects using Predictive Simulation and Trajectory Optimization](https://mlsanity.com/paper/1512.06922/) (2015)
- [Learning Task-Parameterized Skills from Few Demonstrations](https://mlsanity.com/paper/2201.09975/) (2022)
- [Do You Need a Hand? -- a Bimanual Robotic Dressing Assistance Scheme](https://mlsanity.com/paper/2301.02749/) (2023)
- [RaggeDi: Diffusion-based State Estimation of Disordered Rags, Sheets, Towels and Blankets](https://mlsanity.com/paper/2409.11831/) (2024)

## Built on by (most cited first, 3 of 3)

- [Robustness without Wrinkles: Parallel Simulation and Robust MPC for Certified Deformable Manipulation](https://mlsanity.com/paper/2606.14188/) (2026)
- [PPO-EAL: Exact Augmented Lagrangian Proximal Policy Optimization for Safe Robotic Control](https://mlsanity.com/paper/2606.27861/) (2026)
- [Dynamics-Aware Meta-Imitation for Generalization to Unseen Robotic Manipulation](https://mlsanity.com/paper/2607.15880/) (2026)

## Abstract (arXiv)

In this work, we conducted research on deformable object manipulation by robots based on demonstration-enhanced reinforcement learning (RL). To improve the learning efficiency of RL, we enhanced the utilization of demonstration data from multiple aspects and proposed the HGCR-DDPG algorithm. It uses a novel high-dimensional fuzzy approach for grasping-point selection, a refined behavior-cloning method to enhance data-driven learning in Rainbow-DDPG, and a sequential policy-learning strategy. Compared to the baseline algorithm (Rainbow-DDPG), our proposed HGCR-DDPG achieved 2.01 times the global average reward and reduced the global average standard deviation to 45% of that of the baseline algorithm. To reduce the human labor cost of demonstration collection, we proposed a low-cost demonstration collection method based on Nonlinear Model Predictive Control (NMPC). Simulation experiment results show that demonstrations collected through NMPC can be used to train HGCR-DDPG, achieving comparable results to those obtained with human demonstrations. To validate the feasibility of our proposed methods in real-world environments, we conducted physical experiments involving deformable object manipulation. We manipulated fabric to perform three tasks: diagonal folding, central axis folding, and flattening. The experimental results demonstrate that our proposed method achieved success rates of 83.3%, 80%, and 100% for these three tasks, respectively, validating the effectiveness of our approach. Compared to current large-model approaches for robot manipulation, the proposed algorithm is lightweight, requires fewer computational resources, and offers task-specific customization and efficient adaptability for specific tasks.