mlsanityData to 7 Oct 2026

DexPoint: Generalizable Point Cloud Reinforcement Learning for Sim-to-Real Dexterous Manipulation

arXiv 2211.09423 ↗First posted 17 Nov 2022cs.RO cs.CV cs.LGcited by 78

Not on the map. Why?

Abstract (arXiv)

We propose a sim-to-real framework for dexterous manipulation which can generalize to new objects of the same category in the real world. The key of our framework is to train the manipulation policy with point cloud inputs and dexterous hands. We propose two new techniques to enable joint learning on multiple objects and sim-to-real generalization: (i) using imagined hand point clouds as augmented inputs; and (ii) designing novel contact-based rewards. We empirically evaluate our method using an Allegro Hand to grasp novel objects in both simulation and real world. To the best of our knowledge, this is the first policy learning-based framework that achieves such generalization results with dexterous hands. Our project page is available at https://yzqin.github.io/dexpoint

Key facts. Cited by 78 papers in the map (data through 7 Oct 2026). Builds on 40 of its 65 references that are in the map. First posted on arXiv 17 Nov 2022.

Builds on

40 of 65 references.

References · 40

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  3. 2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 685
  4. 2018Learning Dexterous In-Hand Manipulationcited by 643
  5. 2020SAPIEN: A SimulAted Part-based Interactive ENvironmentcited by 584
  6. 2017Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperationcited by 286
  7. 2017Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metricscited by 259
  8. 20196-DOF GraspNet: Variational Grasp Generation for Object Manipulationcited by 237
  9. 2021DexMV: Imitation Learning for Dexterous Manipulation from Human Videoscited by 194
  10. 2021ManiSkill: Generalizable Manipulation Skill Benchmark with Large-Scale Demonstrationscited by 193
  11. 2020DIGIT: A Novel Design for a Low-Cost Compact High-Resolution Tactile Sensor with Application to In-Hand Manipulationcited by 164
  12. 2021A System for General In-Hand Object Re-Orientationcited by 155
28 more
  1. 2019A Review of Robot Learning for Manipulation: Challenges, Representations, and Algorithmscited by 136
  2. 2017Grasp Pose Detection in Point Cloudscited by 107
  3. 2013Data-Driven Grasp Synthesis - A Surveycited by 104
  4. 2018More Than a Feeling: Learning to Grasp and Regrasp using Vision and Touchcited by 100
  5. 2021Coarse-to-Fine Imitation Learning: Robot Manipulation from a Single Demonstrationcited by 92
  6. 2020State-Only Imitation Learning for Dexterous Manipulationcited by 90
  7. 2019ContactGrasp: Functional Multi-finger Grasp Synthesis from Contactcited by 87
  8. 2018PointNetGPD: Detecting Grasp Configurations from Point Setscited by 80
  9. 2019Making Sense of Vision and Touch: Learning Multimodal Representations for Contact-Rich Taskscited by 80
  10. 2018Dexterous Manipulation with Deep Reinforcement Learning: Efficient, General, and Low-Costcited by 79
  11. 2022From One Hand to Multiple Hands: Imitation Learning for Dexterous Manipulation from Single-Camera Teleoperationcited by 79
  12. 2019S4G: Amodal Single-view Single-Shot SE(3) Grasp Detection in Cluttered Scenescited by 51
  13. 2021Generalization in Dexterous Manipulation via Geometry-Aware Multi-Task Learningcited by 49
  14. 2022Learning Generalizable Dexterous Manipulation from Human Grasp Affordancecited by 46
  15. 2020PointNet++ Grasping: Learning An End-to-end Spatial Grasp Generation Algorithm from Sparse Point Cloudscited by 39
  16. 2015Learning Contact-Rich Manipulation Skills with Guided Policy Searchcited by 37
  17. 2016Learning Dexterous Manipulation for a Soft Robotic Hand from Human Demonstrationcited by 35
  18. 2021ReSkin: versatile, replaceable, lasting tactile skinscited by 33
  19. 2019Learning Gentle Object Manipulation with Curiosity-Driven Deep Reinforcement Learningcited by 25
  20. 2020Goal-Auxiliary Actor-Critic for 6D Robotic Grasping with Point Cloudscited by 24
  21. 2020Learning Dexterous Manipulation from Suboptimal Expertscited by 23
  22. 2020Grasp Proposal Networks: An End-to-End Solution for Visual Learning of Robotic Graspscited by 22
  23. 2022Learning Robust Real-World Dexterous Grasping Policies via Implicit Shape Augmentationcited by 21
  24. 2018Planning Multi-Fingered Grasps as Probabilistic Inference in a Learned Deep Networkcited by 19
  25. 2018Learning to Grasp Without Seeingcited by 19
  26. 2021Towards Learning to Play Piano with Dexterous Hands and Touchcited by 14
  27. 2021GPR: Grasp Pose Refinement Network for Cluttered Scenescited by 13
  28. 2021Improved Learning of Robot Manipulation Tasks via Tactile Intrinsic Motivationcited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 20; 2024: 29; 2025: 12; 2026: 17.

78 papers.

Sort:

By area:

  1. 2024DexCap: Scalable and Portable Mocap Data Collection System for Dexterous Manipulationcited by 149
  2. 2023AnyTeleop: A General Vision-Based Dexterous Robot Arm-Hand Teleoperation Systemcited by 136
  3. 2024Flow as the Cross-Domain Manipulation Interfacecited by 126
  4. 2023UniDexGrasp: Universal Robotic Dexterous Grasping via Learning Diverse Proposal Generation and Goal-Conditioned Policycited by 117
  5. 2023UniDexGrasp++: Improving Dexterous Grasping Policy Learning via Geometry-aware Curriculum and Iterative Generalist-Specialist Learningcited by 101
  6. 2023General In-Hand Object Rotation with Vision and Touchcited by 87
  7. 2023Rotating without Seeing: Towards In-hand Dexterity through Touchcited by 78
  8. 2024Bunny-VisionPro: Real-Time Bimanual Dexterous Teleoperation for Imitation Learningcited by 69
  9. 20243D-ViTac: Learning Fine-Grained Manipulation with Visuo-Tactile Sensingcited by 67
  10. 2023DexArt: Benchmarking Generalizable Dexterous Manipulation with Articulated Objectscited by 64
  11. 2024GraspXL: Generating Grasping Motions for Diverse Objects at Scalecited by 59
  12. 2023PolarNet: 3D Point Clouds for Language-Guided Robotic Manipulationcited by 56
38 more
  1. 2023ArtiGrasp: Physically Plausible Synthesis of Bi-Manual Dexterous Grasping and Articulationcited by 53
  2. 2023Stanford-ORB: A Real-World 3D Object Inverse Rendering Benchmarkcited by 51
  3. 2023Robot Synesthesia: In-Hand Manipulation with Visuotactile Sensingcited by 51
  4. 2024TRANSIC: Sim-to-Real Policy Transfer by Learning from Online Correctioncited by 50
  5. 2024DexDiffuser: Generating Dexterous Grasps with Diffusion Modelscited by 46
  6. 2023Sequential Dexterity: Chaining Dexterous Policies for Long-Horizon Manipulationcited by 43
  7. 2023Dexterous Functional Graspingcited by 36
  8. 2023Dynamic Handover: Throw and Catch with Bimanual Handscited by 33
  9. 2024RealDex: Towards Human-like Grasping for Robotic Dexterous Handcited by 33
  10. 2024Adaptive Mobile Manipulation for Articulated Objects In the Open Worldcited by 30
  11. 2025Touch in the Wild: Learning Fine-Grained Manipulation with a Portable Visuo-Tactile Grippercited by 29
  12. 2023HACMan: Learning Hybrid Actor-Critic Maps for 6D Non-Prehensile Manipulationcited by 27
  13. 2024GenDP: 3D Semantic Fields for Category-Level Generalizable Diffusion Policycited by 25
  14. 2024Cross-Embodiment Dexterous Grasping with Reinforcement Learningcited by 24
  15. 2023GraspGF: Learning Score-based Grasping Primitive for Human-assisting Dexterous Graspingcited by 23
  16. 2023SparseDFF: Sparse-View Feature Distillation for One-Shot Dexterous Manipulationcited by 22
  17. 2024Lessons from Learning to Spin "Pens"cited by 22
  18. 2026LLM-as-a-Verifier: A General-Purpose Verification Frameworkcited by 21
  19. 2023A Universal Semantic-Geometric Representation for Robotic Manipulationcited by 16
  20. 2023Harmonic Mobile Manipulationcited by 16
  21. 2024Point Cloud Models Improve Visual Robustness in Robotic Learnerscited by 13
  22. 2024Leveraging Locality to Boost Sample Efficiency in Robotic Manipulationcited by 13
  23. 2024Learning Diverse Bimanual Dexterous Manipulation Skills from Human Demonstrationscited by 12
  24. 2024DexTouch: Learning to Seek and Manipulate Objects with Tactile Dexteritycited by 11
  25. 2024Reinforcement Learning with Generalizable Gaussian Splattingcited by 11
  26. 2024QuasiSim: Parameterized Quasi-Physical Simulators for Dexterous Manipulations Transfercited by 10
  27. 2025Web2Grasp: Learning Functional Grasps from Web Images of Hand-Object Interactionscited by 10
  28. 2025VT-Refine: Learning Bimanual Assembly with Visuo-Tactile Feedback via Simulation Fine-Tuningcited by 10
  29. 2024Part-Guided 3D RL for Sim2Real Articulated Object Manipulationcited by 9
  30. 2024Efficient Residual Learning with Mixture-of-Experts for Universal Dexterous Graspingcited by 8
  31. 2024Planning-Guided Diffusion Policy Learning for Generalizable Contact-Rich Bimanual Manipulationcited by 8
  32. 2025From Simple to Complex Skills: The Case of In-Hand Object Reorientationcited by 8
  33. 2024Hand-Centric Motion Refinement for 3D Hand-Object Interaction via Hierarchical Spatial-Temporal Modelingcited by 6
  34. 2024MATCH POLICY: A Simple Pipeline from Point Cloud Registration to Manipulation Policiescited by 6
  35. 2023Robust Visual Sim-to-Real Transfer for Robotic Manipulationcited by 5
  36. 2024PointPatchRL -- Masked Reconstruction Improves Reinforcement Learning on Point Cloudscited by 5
  37. 2024Neural Attention Field: Emerging Point Relevance in 3D Scenes for One-Shot Dexterous Graspingcited by 5
  38. 2025ManiVID-3D: Generalizable View-Invariant Reinforcement Learning for Robotic Manipulation via Disentangled 3D Representationscited by 5

50 most cited of 78 All 78 (JSON)