mlsanityData to 14 Oct 2026

On the Efficacy of 3D Point Cloud Reinforcement Learning

arXiv 2306.06799 ↗First posted 11 Jun 2023cs.RO cs.AI cs.LGcited by 11

Not on the map. Why?

Abstract (arXiv)

Recent studies on visual reinforcement learning (visual RL) have explored the use of 3D visual representations. However, none of these work has systematically compared the efficacy of 3D representations with 2D representations across different tasks, nor have they analyzed 3D representations from the perspective of agent-object / object-object relationship reasoning. In this work, we seek answers to the question of when and how do 3D neural networks that learn features in the 3D-native space provide a beneficial inductive bias for visual RL. We specifically focus on 3D point clouds, one of the most common forms of 3D representations. We systematically investigate design choices for 3D point cloud RL, leading to the development of a robust algorithm for various robotic manipulation and control tasks. Furthermore, through comparisons between 2D image vs 3D point cloud RL methods on both minimalist synthetic tasks and complex robotic manipulation tasks, we find that 3D point cloud RL can significantly outperform the 2D counterpart when agent-object / object-object relationship encoding is a key factor.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Builds on 55 of its 62 references that are in the map. First posted on arXiv 11 Jun 2023.

Builds on

55 of 62 references.

References · 55

  1. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,248
  2. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,401
  3. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  4. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
  5. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,498
  6. 2020Point Transformercited by 1,470
  7. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,264
  8. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,240
  9. 20194D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networkscited by 1,208
  10. 2019KPConv: Flexible and Deformable Convolution for Point Cloudscited by 1,205
  11. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,103
  12. 2020Mastering Atari with Discrete World Modelscited by 1,007
38 more
  1. 2018DeepMind Control Suitecited by 956
  2. 2022A Generalist Agentcited by 808
  3. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 760
  4. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 743
  5. 2022R3M: A Universal Visual Representation for Robot Manipulationcited by 694
  6. 2017Sim-to-Real Transfer of Robotic Control with Dynamics Randomizationcited by 672
  7. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 585
  8. 2022Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulationcited by 551
  9. 2022BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learningcited by 523
  10. 2020Reinforcement Learning with Augmented Datacited by 468
  11. 2018Pseudo-LiDAR from Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Drivingcited by 457
  12. 2020Searching Efficient 3D Architectures with Sparse Point-Voxel Convolutioncited by 423
  13. 2022Temporal Difference Learning for Model Predictive Controlcited by 397
  14. 2021Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learningcited by 372
  15. 2020Planning to Explore via Self-Supervised World Modelscited by 368
  16. 2019DenseFusion: 6D Object Pose Estimation by Iterative Dense Fusioncited by 355
  17. 2020Transporter Networks: Rearranging the Visual World for Robotic Manipulationcited by 344
  18. 2019Improving Sample Efficiency in Model-Free Reinforcement Learning from Imagescited by 274
  19. 2019RoboNet: Large-Scale Multi-Robot Learningcited by 270
  20. 2022Masked Visual Pre-training for Motor Controlcited by 248
  21. 2017Asymmetric Actor Critic for Image-Based Robot Learningcited by 238
  22. 2021Contact-GraspNet: Efficient 6-DoF Grasp Generation in Cluttered Scenescited by 218
  23. 2021Neural Descriptor Fields: SE(3)-Equivariant Object Representations for Manipulationcited by 209
  24. 2019PVN3D: A Deep Point-wise 3D Keypoints Voting Network for 6DoF Pose Estimationcited by 204
  25. 2022SoftGroup for 3D Instance Segmentation on Point Cloudscited by 200
  26. 2021ManiSkill: Generalizable Manipulation Skill Benchmark with Large-Scale Demonstrationscited by 197
  27. 2019Active Domain Randomizationcited by 170
  28. 2021A System for General In-Hand Object Re-Orientationcited by 165
  29. 2019Mastering Complex Control in MOBA Games with Deep Reinforcement Learningcited by 161
  30. 2022The Unsurprising Effectiveness of Pre-Trained Vision Models for Controlcited by 155
  31. 2021FFB6D: A Full Flow Bidirectional Fusion Network for 6D Pose Estimationcited by 144
  32. 2018Reinforcement and Imitation Learning for Diverse Visuomotor Skillscited by 126
  33. 2021Coarse-to-Fine Q-attention: Efficient Learning for Visual Robotic Manipulation via Discretisationcited by 122
  34. 2021Stabilizing Deep Q-Learning with ConvNets and Vision Transformers under Data Augmentationcited by 105
  35. 2022FlowBot3D: Learning 3D Articulation Flow to Manipulate Articulated Objectscited by 91
  36. 2021AdaAfford: Learning to Adapt Manipulation Affordance for 3D Articulated Objects via Few-shot Interactionscited by 57
  37. 2022Learning Generalizable Dexterous Manipulation from Human Grasp Affordancecited by 51
  38. 2021Generalization in Dexterous Manipulation via Geometry-Aware Multi-Task Learningcited by 50

50 most cited of 55 All 55 (JSON)

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 7; 2025: 3.

11 papers.

  1. 2024Grounding Video Models to Actions through Goal Conditioned Explorationcited by 26
  2. 2024Point Cloud Models Improve Visual Robustness in Robotic Learnerscited by 14
  3. 2024Reinforcement Learning with Generalizable Gaussian Splattingcited by 13
  4. 2023Robo360: A 3D Omnispective Multi-Material Robotic Manipulation Datasetcited by 9
  5. 2024PointPatchRL -- Masked Reconstruction Improves Reinforcement Learning on Point Cloudscited by 5
  6. 2024BEVNav: Robot Autonomous Navigation Via Spatial-Temporal Contrastive Learning in Bird's-Eye Viewcited by 3
  7. 2025Robust Point Cloud Reinforcement Learning via PCA-Based Canonicalizationcited by 1
  8. 2024Learning the Generalizable Manipulation Skills on Soft-body Tasks via Guided Self-attention Behavior Cloning Policycited by 0
  9. 2024GSL-PCD: Improving Generalist-Specialist Learning with Point Cloud Feature-based Task Partitioningcited by 0
  10. 2025A comparison of visual representations for real-world reinforcement learning in the context of vacuum grippingcited by 0
  11. 2025Deformable Cluster Manipulation via Whole-Arm Policy Learningcited by 0