# TWISTED-RL: Hierarchical Skilled Agents for Knot-Tying without Human Demonstrations

arXiv: https://arxiv.org/abs/2602.14526 · HTML: https://mlsanity.com/paper/2602.14526/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 23 references that are in the map.
- First posted on arXiv 16 Feb 2026.

## Builds on (most cited first, 12 of 12)

- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Go-Explore: a New Approach for Hard-Exploration Problems](https://mlsanity.com/paper/1901.10995/) (2019)
- [Sim-to-Real Reinforcement Learning for Deformable Object Manipulation](https://mlsanity.com/paper/1806.07851/) (2018)
- [SoftGym: Benchmarking Deep Reinforcement Learning for Deformable Object Manipulation](https://mlsanity.com/paper/2011.07215/) (2020)
- [Combining Self-Supervised Learning and Imitation for Vision-Based Rope Manipulation](https://mlsanity.com/paper/1703.02018/) (2017)
- [Learning to Manipulate Deformable Objects without Demonstrations](https://mlsanity.com/paper/1910.13439/) (2019)
- [Self-Supervised Learning of State Estimation for Manipulating Deformable Linear Objects](https://mlsanity.com/paper/1911.06283/) (2019)
- [Challenges and Outlook in Robotic Manipulation of Deformable Objects](https://mlsanity.com/paper/2105.01767/) (2021)
- [Deep Reinforcement Learning Based on Local GNN for Goal-conditioned Deformable Object Rearranging](https://mlsanity.com/paper/2302.10446/) (2023)
- [A Survey on Robotic Manipulation of Deformable Objects: Recent Advances, Open Challenges and New Frontiers](https://mlsanity.com/paper/2312.10419/) (2023)
- [Learning Topological Motion Primitives for Knot Planning](https://mlsanity.com/paper/2009.02615/) (2020)
- [SGTM 2.0: Autonomously Untangling Long Cables using Interactive Perception](https://mlsanity.com/paper/2209.13706/) (2022)

## Abstract (arXiv)

Robotic knot-tying represents a fundamental challenge in robotics due to the complex interactions between deformable objects and strict topological constraints. We present TWISTED-RL, a framework that improves upon the previous state-of-the-art in demonstration-free knot-tying (TWISTED), which smartly decomposed a single knot-tying problem into manageable subproblems, each addressed by a specialized agent. Our approach replaces TWISTED's single-step inverse model that was learned via supervised learning with a multi-step Reinforcement Learning policy conditioned on abstract topological actions rather than goal states. This change allows more delicate topological state transitions while avoiding costly and ineffective data collection protocols, thus enabling better generalization across diverse knot configurations. Experimental results demonstrate that TWISTED-RL manages to solve previously unattainable knots of higher complexity, including commonly used knots such as the Figure-8 and the Overhand. Furthermore, the increase in success rates and drop in planning time establishes TWISTED-RL as the new state-of-the-art in robotic knot-tying without human demonstrations.