mlsanityData to 14 Oct 2026

Dynamic Handover: Throw and Catch with Bimanual Hands

arXiv 2309.05655 ↗First posted 11 Sep 2023cs.RO cs.LGcited by 35

Not on the map. Why?

Abstract (arXiv)

Humans throw and catch objects all the time. However, such a seemingly common skill introduces a lot of challenges for robots to achieve: The robots need to operate such dynamic actions at high-speed, collaborate precisely, and interact with diverse objects. In this paper, we design a system with two multi-finger hands attached to robot arms to solve this problem. We train our system using Multi-Agent Reinforcement Learning in simulation and perform Sim2Real transfer to deploy on the real robots. To overcome the Sim2Real gap, we provide multiple novel algorithm designs including learning a trajectory prediction model for the object. Such a model can help the robot catcher has a real-time estimation of where the object will be heading, and then react accordingly. We conduct our experiments with multiple objects in the real-world system, and show significant improvements over multiple baselines. Our project page is available at urlhttps://binghao-huang.github.io/dynamichandover/.

Key facts. Cited by 35 papers in the map (data through 14 Oct 2026). Builds on 31 of its 52 references that are in the map. First posted on arXiv 11 Sep 2023.

Builds on

31 of 52 references.

References · 31

  1. 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 998
  2. 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 942
  3. 2017Counterfactual Multi-Agent Policy Gradientscited by 751
  4. 2019Solving Rubik's Cube with a Robot Handcited by 734
  5. 2018Learning Dexterous In-Hand Manipulationcited by 687
  6. 2016Learning to Communicate with Deep Multi-Agent Reinforcement Learningcited by 575
  7. 2019QTRAN: Learning to Factorize with Transformation for Cooperative Multi-Agent Reinforcement Learningcited by 379
  8. 2021A System for General In-Hand Object Re-Orientationcited by 165
  9. 2022DeXtreme: Transfer of Agile In-hand Manipulation from Simulation to Realitycited by 149
  10. 2019TossingBot: Learning to Throw Arbitrary Objects with Residual Physicscited by 137
  11. 2020ROMA: Multi-Agent Reinforcement Learning with Emergent Rolescited by 137
  12. 2022Towards Human-Level Bimanual Dexterous Manipulation with Reinforcement Learningcited by 119
19 more
  1. 2022In-Hand Object Rotation via Rapid Motor Adaptationcited by 118
  2. 2013Data-Driven Grasp Synthesis - A Surveycited by 106
  3. 2016Opponent Modeling in Deep Reinforcement Learningcited by 97
  4. 2018Dexterous Manipulation with Deep Reinforcement Learning: Efficient, General, and Low-Costcited by 84
  5. 2022DexPoint: Generalizable Point Cloud Reinforcement Learning for Sim-to-Real Dexterous Manipulationcited by 84
  6. 2023Rotating without Seeing: Towards In-hand Dexterity through Touchcited by 84
  7. 2022From One Hand to Multiple Hands: Imitation Learning for Dexterous Manipulation from Single-Camera Teleoperationcited by 82
  8. 2019Efficient Bimanual Manipulation Using Learned Task Schemascited by 45
  9. 2019Multi-Agent Manipulation via Locomotion using Hierarchical Sim2Realcited by 42
  10. 2022Iterative Residual Policy: for Goal-Conditioned Dynamic Manipulation of Deformable Objectscited by 37
  11. 2016Learning Dexterous Manipulation for a Soft Robotic Hand from Human Demonstrationcited by 36
  12. 2022Learning Continuous Grasping Function with a Dexterous Hand from Human Demonstrationscited by 34
  13. 2017Deep Predictive Policy Training using Reinforcement Learningcited by 32
  14. 2023DexPBT: Scaling up Dexterous Manipulation for Hand-Arm Systems with Population Based Trainingcited by 29
  15. 2021SwingBot: Learning Physical Features from In-hand Tactile Exploration for Dynamic Swing-up Manipulationcited by 24
  16. 2023DexDeform: Dexterous Deformable Object Manipulation with Human Demonstrations and Differentiable Physicscited by 23
  17. 2022Bi-Manual Manipulation and Attachment via Sim-to-Real Reinforcement Learningcited by 14
  18. 2020Distributed Reinforcement Learning for Cooperative Multi-Robot Object Manipulationcited by 7
  19. 2021An Unconstrained Convex Formulation of Compliant Contactcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 15; 2025: 14; 2026: 4.

35 papers.

Sort:

By area:

  1. 2024DexCap: Scalable and Portable Mocap Data Collection System for Dexterous Manipulationcited by 160
  2. 20243D Diffusion Policy: Generalizable Visuomotor Policy Learning via Simple 3D Representationscited by 116
  3. 2025DexGraspVLA: A Vision-Language-Action Framework Towards General Dexterous Graspingcited by 89
  4. 2024Learning Visuotactile Skills with Two Multifingered Handscited by 76
  5. 20243D-ViTac: Learning Fine-Grained Manipulation with Visuo-Tactile Sensingcited by 72
  6. 2024SplatSim: Zero-Shot Sim2Real Transfer of RGB Manipulation Policies Using Gaussian Splattingcited by 69
  7. 2025ManipTrans: Efficient Dexterous Bimanual Manipulation Transfer via Residual Learningcited by 66
  8. 2023Robot Synesthesia: In-Hand Manipulation with Visuotactile Sensingcited by 56
  9. 2025Sim-to-Real Reinforcement Learning for Vision-Based Dexterous Manipulation on Humanoidscited by 48
  10. 2024Learning to Manipulate Anywhere: A Visual Generalizable Framework For Reinforcement Learningcited by 41
  11. 2024BimArt: A Unified Approach for the Synthesis of 3D Bimanual Interaction with Articulated Objectscited by 30
  12. 2025Touch in the Wild: Learning Fine-Grained Manipulation with a Portable Visuo-Tactile Grippercited by 29
23 more
  1. 2024CyberDemo: Augmenting Simulated Human Demonstration for Real-World Dexterous Manipulationcited by 28
  2. 2024Lessons from Learning to Spin "Pens"cited by 26
  3. 2024VoxAct-B: Voxel-Based Acting and Stabilizing Policy for Bimanual Manipulationcited by 21
  4. 2024AIC MLLM: Autonomous Interactive Correction MLLM for Robust Robotic Manipulationcited by 20
  5. 2024Learning Diverse Bimanual Dexterous Manipulation Skills from Human Demonstrationscited by 15
  6. 2024Learning Multi-Agent Loco-Manipulation for Long-Horizon Quadrupedal Pushingcited by 13
  7. 2025VT-Refine: Learning Bimanual Assembly with Visuo-Tactile Feedback via Simulation Fine-Tuningcited by 11
  8. 2025Learning Dexterous Manipulation Skills from Imperfect Simulationscited by 9
  9. 2023DexCatch: Learning to Catch Arbitrary Objects with Dexterous Handscited by 8
  10. 2025From Simple to Complex Skills: The Case of In-Hand Object Reorientationcited by 8
  11. 2025A Real-to-Sim-to-Real Approach to Robotic Manipulation with VLM-Generated Iterative Keypoint Rewardscited by 5
  12. 2025Morphologically Symmetric Reinforcement Learning for Ambidextrous Bimanual Manipulationcited by 5
  13. 2024Dexterous Functional Pre-Grasp Manipulation with Diffusion Policycited by 4
  14. 2025VTAO-BiManip: Masked Visual-Tactile-Action Pre-training with Object Understanding for Bimanual Dexterous Manipulationcited by 3
  15. 2025GauDP: Reinventing Multi-Agent Collaboration through Gaussian-Image Synergy in Diffusion Policiescited by 2
  16. 2025Learning Dexterous Bimanual Catch Skills through Adversarial-Cooperative Heterogeneous-Agent Reinforcement Learningcited by 1
  17. 2026TEMPO: Learning Temporal Context for Dynamic Robot Manipulationcited by 1
  18. 2024AsymDex: Asymmetry and Relative Coordinates for RL-based Bimanual Dexteritycited by 0
  19. 2025SpikePingpong: Spike Vision-based Fast-Slow Pingpong Robot Systemcited by 0
  20. 2025Learning Dexterous Object Handovercited by 0
  21. 2026Cooperative-Competitive Team Play of Real-World Craft Robotscited by 0
  22. 2026DCGrasp: Distance-aware Controllable Grasp Generationcited by 0
  23. 2026Directional Constraints for Efficient Exploration in Safe Reinforcement Learningcited by 0