mlsanityData to 7 Oct 2026

Generalization in Dexterous Manipulation via Geometry-Aware Multi-Task Learning

arXiv 2111.03062 ↗First posted 4 Nov 2021cs.RO cs.AI cs.CVcited by 49

Not on the map. Why?

Abstract (arXiv)

Dexterous manipulation of arbitrary objects, a fundamental daily task for humans, has been a grand challenge for autonomous robotic systems. Although data-driven approaches using reinforcement learning can develop specialist policies that discover behaviors to control a single object, they often exhibit poor generalization to unseen ones. In this work, we show that policies learned by existing reinforcement learning algorithms can in fact be generalist when combined with multi-task learning and a well-chosen object representation. We show that a single generalist policy can perform in-hand manipulation of over 100 geometrically-diverse real-world objects and generalize to new objects with unseen shape or size. Interestingly, we find that multi-task learning with object point cloud representations not only generalizes better but even outperforms the single-object specialist policies on both training as well as held-out test objects. Video results at https://huangwl18.github.io/geometry-dex

Key facts. Cited by 49 papers in the map (data through 7 Oct 2026). Builds on 29 of its 40 references that are in the map. First posted on arXiv 4 Nov 2021.

Builds on

29 of 40 references.

References · 29

  1. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  2. 2015Continuous control with deep reinforcement learningcited by 3,279
  3. 2016OpenAI Gymcited by 1,684
  4. 2020Gradient Surgery for Multi-Task Learningcited by 1,210
  5. 2017Hindsight Experience Replaycited by 1,096
  6. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 866
  7. 2015End-to-End Training of Deep Visuomotor Policiescited by 847
  8. 2019Solving Rubik's Cube with a Robot Handcited by 699
  9. 2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 685
  10. 2018Learning Dexterous In-Hand Manipulationcited by 643
  11. 2016RL2: Fast Reinforcement Learning via Slow Reinforcement Learningcited by 490
  12. 2019Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variablescited by 372
17 more
  1. 2016Learning to reinforcement learncited by 369
  2. 2018Multi-Goal Reinforcement Learning: Challenging Robotics Environments and Request for Researchcited by 256
  3. 2017Distral: Robust Multitask Reinforcement Learningcited by 231
  4. 2019Deep Dynamics Models for Learning Dexterous Manipulationcited by 200
  5. 2017Emergence of Locomotion Behaviours in Rich Environmentscited by 159
  6. 2019ContactDB: Analyzing and Predicting Grasp Contact via Thermal Imagingcited by 145
  7. 2018Multi-task Deep Reinforcement Learning with PopArtcited by 138
  8. 2020State-Only Imitation Learning for Dexterous Manipulationcited by 90
  9. 2018Discovery of Latent 3D Keypoints via End-to-end Geometric Reasoningcited by 79
  10. 2018Dexterous Manipulation with Deep Reinforcement Learning: Efficient, General, and Low-Costcited by 79
  11. 2016Learning Dexterous Manipulation for a Soft Robotic Hand from Human Demonstrationcited by 35
  12. 2016Learning Dexterous Manipulation Policies from Experience and Imitationcited by 32
  13. 2020Learning Dexterous Manipulation from Suboptimal Expertscited by 23
  14. 2020Solving Challenging Dexterous Manipulation Tasks With Trajectory Optimisation and Reinforcement Learningcited by 12
  15. 2018In-Hand Manipulation via Motion Conescited by 9
  16. 2018Geometric In-Hand Regrasp Planning: Alternating Optimization of Finger Gaits and In-Grasp Manipulationcited by 6
  17. 2017Experimental Validation of Contact Dynamics for In-Hand Manipulationcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 16; 2023: 17; 2024: 8; 2025: 4; 2026: 3.

49 papers.

Sort:

By area:

  1. 2022Masked Visual Pre-training for Motor Controlcited by 246
  2. 2023RVT: Robotic View Transformer for 3D Object Manipulationcited by 224
  3. 2022DeXtreme: Transfer of Agile In-hand Manipulation from Simulation to Realitycited by 136
  4. 2023UniDexGrasp: Universal Robotic Dexterous Grasping via Learning Diverse Proposal Generation and Goal-Conditioned Policycited by 117
  5. 2022Towards Human-Level Bimanual Dexterous Manipulation with Reinforcement Learningcited by 113
  6. 2021D-Grasp: Physically Plausible Dynamic Grasp Synthesis for Hand-Object Interactionscited by 109
  7. 2022In-Hand Object Rotation via Rapid Motor Adaptationcited by 102
  8. 2023UniDexGrasp++: Improving Dexterous Grasping Policy Learning via Geometry-aware Curriculum and Iterative Generalist-Specialist Learningcited by 101
  9. 2023Toward General-Purpose Robots via Foundation Models: A Survey and Meta-Analysiscited by 99
  10. 2023General In-Hand Object Rotation with Vision and Touchcited by 87
  11. 2022MyoSuite -- A contact-rich simulation suite for musculoskeletal motor controlcited by 80
  12. 2022DexPoint: Generalizable Point Cloud Reinforcement Learning for Sim-to-Real Dexterous Manipulationcited by 78
37 more
  1. 2023LEAP Hand: Low-Cost, Efficient, and Anthropomorphic Hand for Robot Learningcited by 78
  2. 2023DexArt: Benchmarking Generalizable Dexterous Manipulation with Articulated Objectscited by 64
  3. 2023Dexterity from Touch: Self-Supervised Pre-Training of Tactile Representations with Robotic Playcited by 60
  4. 2022Holo-Dex: Teaching Dexterity with Immersive Mixed Realitycited by 52
  5. 2022Learning Generalizable Dexterous Manipulation from Human Grasp Affordancecited by 46
  6. 2023Sequential Dexterity: Chaining Dexterous Policies for Long-Horizon Manipulationcited by 43
  7. 2022Learning Continuous Grasping Function with a Dexterous Hand from Human Demonstrationscited by 34
  8. 2024RealDex: Towards Human-like Grasping for Robotic Dexterous Handcited by 33
  9. 2022Frame Mining: a Free Lunch for Learning Robotic Manipulation from 3D Point Cloudscited by 31
  10. 2022Learning Dexterous Manipulation from Exemplar Object Trajectories and Pre-Graspscited by 27
  11. 2023HACMan: Learning Hybrid Actor-Critic Maps for 6D Non-Prehensile Manipulationcited by 27
  12. 2022PACT: Perception-Action Causal Transformer for Autoregressive Robotics Pre-Trainingcited by 24
  13. 2025Bigger, Regularized, Categorical: High-Capacity Value Functions are Efficient Multi-Task Learnerscited by 23
  14. 2023SparseDFF: Sparse-View Feature Distillation for One-Shot Dexterous Manipulationcited by 22
  15. 2022Blocks Assemble! Learning to Assemble with Large-Scale Structured Reinforcement Learningcited by 21
  16. 2023A Universal Semantic-Geometric Representation for Robotic Manipulationcited by 16
  17. 2024DexHandDiff: Interaction-aware Diffusion Planning for Adaptive Dexterous Manipulationcited by 15
  18. 2025DEXOP: A Device for Robotic Transfer of Dexterous Human Manipulationcited by 15
  19. 2022Bi-Manual Manipulation and Attachment via Sim-to-Real Reinforcement Learningcited by 13
  20. 2024Leveraging Locality to Boost Sample Efficiency in Robotic Manipulationcited by 13
  21. 2024Learning by Watching: A Review of Video-based Learning Approaches for Robot Manipulationcited by 10
  22. 2023On the Efficacy of 3D Point Cloud Reinforcement Learningcited by 9
  23. 2023DexCatch: Learning to Catch Arbitrary Objects with Dexterous Handscited by 8
  24. 2024CORN: Contact-based Object Representation for Nonprehensile Manipulation of General Unseen Objectscited by 8
  25. 2023GRID: A Platform for General Robot Intelligence Developmentcited by 6
  26. 2023Learning to Transfer In-Hand Manipulations Using a Greedy Shape Curriculumcited by 5
  27. 2024PointPatchRL -- Masked Reconstruction Improves Reinforcement Learning on Point Cloudscited by 5
  28. 2023Bi-Manual Block Assembly via Sim-to-Real Reinforcement Learningcited by 4
  29. 2022Efficient Representations of Object Geometry for Reinforcement Learning of Interactive Grasping Policiescited by 2
  30. 2024Diffusion-Informed Probabilistic Contact Search for Multi-Finger Manipulationcited by 2
  31. 2025Robust Point Cloud Reinforcement Learning via PCA-Based Canonicalizationcited by 1
  32. 2025Balance Equation-based Distributionally Robust Offline Imitation Learningcited by 1
  33. 2026Revisiting Mixture Policies in Entropy-Regularized Actor-Criticcited by 1
  34. 2022A Learning System for Motion Planning of Free-Float Dual-Arm Space Manipulator towards Non-Cooperative Objectcited by 0
  35. 2024Learning the Generalizable Manipulation Skills on Soft-body Tasks via Guided Self-attention Behavior Cloning Policycited by 0
  36. 2026When Dynamics Shift, Robust Task Inference Wins: Offline Imitation Learning with Behavior Foundation Models Revisitedcited by 0
  37. 2026Rotation-Aware Point-Cloud Embeddings for Vision-Based In-Hand Reorientationcited by 0