# Deformable Cluster Manipulation via Whole-Arm Policy Learning

arXiv: https://arxiv.org/abs/2507.17085 · HTML: https://mlsanity.com/paper/2507.17085/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 29 of its 51 references that are in the map.
- First posted on arXiv 22 Jul 2025.

## Builds on (most cited first, 29 of 29)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023)
- [Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning](https://mlsanity.com/paper/2108.10470/) (2021)
- [Segment Anything in High Quality](https://mlsanity.com/paper/2306.01567/) (2023)
- [Kernel Mean Embedding of Distributions: A Review and Beyond](https://mlsanity.com/paper/1605.09522/) (2016)
- [Putting the Object Back into Video Object Segmentation](https://mlsanity.com/paper/2310.12982/) (2023)
- [DexPoint: Generalizable Point Cloud Reinforcement Learning for Sim-to-Real Dexterous Manipulation](https://mlsanity.com/paper/2211.09423/) (2022)
- [3D-ViTac: Learning Fine-Grained Manipulation with Visuo-Tactile Sensing](https://mlsanity.com/paper/2410.24091/) (2024)
- [Factory: Fast Contact for Robotic Assembly](https://mlsanity.com/paper/2205.03532/) (2022)
- [ToolFlowNet: Robotic Manipulation with Tools via Predicting Tool Flow from Point Clouds](https://mlsanity.com/paper/2211.09006/) (2022)
- [IndustReal: Transferring Contact-Rich Assembly Tasks from Simulation to Reality](https://mlsanity.com/paper/2305.17110/) (2023)
- [Learning Rope Manipulation Policies Using Dense Object Descriptors Trained on Synthetic Depth Data](https://mlsanity.com/paper/2003.01835/) (2020)
- [Challenges and Outlook in Robotic Manipulation of Deformable Objects](https://mlsanity.com/paper/2105.01767/) (2021)
- [Frame Mining: a Free Lunch for Learning Robotic Manipulation from 3D Point Clouds](https://mlsanity.com/paper/2210.07442/) (2022)
- [Point2Vec for Self-Supervised Representation Learning on Point Clouds](https://mlsanity.com/paper/2303.16570/) (2023)
- [Untangling Dense Knots by Learning Task-Relevant Keypoints](https://mlsanity.com/paper/2011.04999/) (2020)
- [A Bayesian Treatment of Real-to-Sim for Deformable Object Manipulation](https://mlsanity.com/paper/2112.05068/) (2021)
- [On the Efficacy of 3D Point Cloud Reinforcement Learning](https://mlsanity.com/paper/2306.06799/) (2023)
- [Planning with Spatial-Temporal Abstraction from Point Clouds for Deformable Object Manipulation](https://mlsanity.com/paper/2210.15751/) (2022)
- [Selecting the State-Representation in Reinforcement Learning](https://mlsanity.com/paper/1302.2552/) (2013)
- [General-purpose Clothes Manipulation with Semantic Keypoints](https://mlsanity.com/paper/2408.08160/) (2024)
- [Manipulation in Clutter with Whole-Arm Tactile Sensing](https://mlsanity.com/paper/1304.6146/) (2013)
- [Learning to Simulate Tree-Branch Dynamics for Manipulation](https://mlsanity.com/paper/2306.03410/) (2023)
- [Towards Robotic Tree Manipulation: Leveraging Graph Representations](https://mlsanity.com/paper/2311.07479/) (2023)
- [Safe Leaf Manipulation for Accurate Shape and Pose Estimation of Occluded Fruits](https://mlsanity.com/paper/2409.17389/) (2024)
- [RetrDex: Efficient Object Retrieval in Cluttered Scenes with a Dexterous Hand](https://mlsanity.com/paper/2502.18423/) (2025)
- [Push Past Green: Learning to Look Behind Plant Foliage by Moving It](https://mlsanity.com/paper/2307.03175/) (2023)
- [Tree Reconstruction using Topology Optimisation](https://mlsanity.com/paper/2205.13192/) (2022)

## Abstract (arXiv)

Manipulating clusters of deformable objects presents a substantial challenge with widespread applicability, but requires contact-rich whole-arm interactions. A potential solution must address the limited capacity for realistic model synthesis, high uncertainty in perception, and the lack of efficient spatial abstractions, among others. We propose a novel framework for learning model-free policies integrating two modalities: 3D point clouds and proprioceptive touch indicators, emphasising manipulation with full body contact awareness, going beyond traditional end-effector modes. Our reinforcement learning framework leverages a distributional state representation, aided by kernel mean embeddings, to achieve improved training efficiency and real-time inference. Furthermore, we propose a novel context-agnostic occlusion heuristic to clear deformables from a target region for exposure tasks. We deploy the framework in a power line clearance scenario and observe that the agent generates creative strategies leveraging multiple arm links for de-occlusion. Finally, we perform zero-shot sim-to-real policy transfer, allowing the arm to clear real branches with unknown occlusion patterns, unseen topology, and uncertain dynamics. Website: https://sites.google.com/view/dcmwap/