mlsanityData to 7 Oct 2026

TieBot: Learning to Knot a Tie from Visual Demonstration through a Real-to-Sim-to-Real Approach

arXiv 2407.03245 ↗First posted 3 Jul 2024cs.RO cs.AI cs.SYcited by 5

Not on the map. Why?

Abstract (arXiv)

The tie-knotting task is highly challenging due to the tie's high deformation and long-horizon manipulation actions. This work presents TieBot, a Real-to-Sim-to-Real learning from visual demonstration system for the robots to learn to knot a tie. We introduce the Hierarchical Feature Matching approach to estimate a sequence of tie's meshes from the demonstration video. With these estimated meshes used as subgoals, we first learn a teacher policy using privileged information. Then, we learn a student policy with point cloud observation by imitating teacher policy. Lastly, our pipeline applies learned policy to real-world execution. We demonstrate the effectiveness of TieBot in simulation and the real world. In the real-world experiment, a dual-arm robot successfully knots a tie, achieving 50% success rate among 10 trials. Videos can be found https://tiebots.github.io/.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 44 of its 53 references that are in the map. First posted on arXiv 3 Jul 2024.

Builds on

44 of 53 references.

References · 44

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  3. 2021LoFTR: Detector-Free Local Feature Matching with Transformerscited by 974
  4. 2022R3M: A Universal Visual Representation for Robot Manipulationcited by 678
  5. 2023CoTracker: It is Better to Track Togethercited by 516
  6. 2023Learning Universal Policies via Text-Guided Video Generationcited by 506
  7. 2017Time-Contrastive Networks: Self-Supervised Learning from Videocited by 453
  8. 2022VIP: Towards Universal Visual Reward and Representation via Value-Implicit Pre-Trainingcited by 384
  9. 2017Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperationcited by 286
  10. 2022Masked Visual Pre-training for Motor Controlcited by 246
  11. 2022Real-World Robot Learning with Masked Visual Pre-trainingcited by 241
  12. 2023Affordances from Human Videos as a Versatile Representation for Roboticscited by 221
32 more
  1. 2023Learning to Act from Actionless Videos through Dense Correspondencescited by 159
  2. 2022Human-to-Robot Imitation in the Wildcited by 157
  3. 2018Residual Policy Learningcited by 146
  4. 2023Structured World Models from Human Videoscited by 144
  5. 2022VideoDex: Learning Dexterity from Internet Videoscited by 101
  6. 2022Visual Dexterity: In-Hand Reorientation of Novel and Complex Object Shapescited by 92
  7. 2023RoboTAP: Tracking Arbitrary Points for Few-Shot Visual Imitationcited by 88
  8. 2021Learning by Watching: Physical Imitation of Manipulation Skills from Human Videoscited by 87
  9. 2021FlingBot: The Unreasonable Effectiveness of Dynamic Manipulation for Cloth Unfoldingcited by 80
  10. 2019Learning to Manipulate Deformable Objects without Demonstrationscited by 76
  11. 2023Video Prediction Models as Rewards for Reinforcement Learningcited by 76
  12. 2024DINO-Tracker: Taming DINO for Self-Supervised Point Tracking in a Single Videocited by 64
  13. 2021FabricFlowNet: Bimanual Cloth Manipulation with a Flow-based Policycited by 54
  14. 2022ToolFlowNet: Robotic Manipulation with Tools via Predicting Tool Flow from Point Cloudscited by 54
  15. 2020Model-Based Inverse Reinforcement Learning from Visual Demonstrationscited by 52
  16. 2022SpeedFolding: Learning Efficient Bimanual Folding of Garmentscited by 51
  17. 2023Zero-Shot Robot Manipulation from Passive Human Videoscited by 49
  18. 2024Vision-based Manipulation from Single Human Video with Open-World Object Graphscited by 46
  19. 2022Graph Inverse Reinforcement Learning from Diverse Videoscited by 45
  20. 2019Self-Supervised Learning of State Estimation for Manipulating Deformable Linear Objectscited by 44
  21. 2019Learning Predictive Models From Observation and Interactioncited by 41
  22. 2021GarmentNets: Category-Level Pose Estimation for Garments via Canonical Space Shape Completioncited by 41
  23. 2022Mesh-based Dynamics with Occlusion Reasoning for Cloth Manipulationcited by 32
  24. 2022SAM-RL: Sensing-Aware Model-Based Reinforcement Learning via Differentiable Physics-Based Simulation and Renderingcited by 22
  25. 2023One Policy to Dress Them All: Learning to Dress People with Diverse Poses and Garmentscited by 18
  26. 2021Occlusion-robust Deformable Object Tracking without Physics Simulationcited by 15
  27. 2023GarmentTracking: Category-Level Garment Pose Trackingcited by 15
  28. 2023Learning Language-Conditioned Deformable Object Manipulation with Graph Dynamicscited by 11
  29. 2023Self-supervised Cloth Reconstruction via Action-conditioned Cloth Trackingcited by 8
  30. 2023TRTM: Template-based Reconstruction and Target-oriented Manipulation of Crumpled Clothscited by 7
  31. 2023Deep Reinforcement Learning Based on Local GNN for Goal-conditioned Deformable Object Rearrangingcited by 6
  32. 2023RangedIK: An Optimization-based Robot Motion Generation Method for Ranged-Goal Taskscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 2.

5 papers.

  1. 2025Robot Learning from Any Imagescited by 7
  2. 2025Large Pre-Trained Models for Bimanual Manipulation in 3Dcited by 1
  3. 2026D-REX: Differentiable Real-to-Sim-to-Real Engine for Learning Dexterous Graspingcited by 1
  4. 2026EgoFun3D: Modeling Interactive Objects from Egocentric Videos using Function Templatescited by 1
  5. 2025DLO-Splatting: Tracking Deformable Linear Objects Using 3D Gaussian Splattingcited by 0