# Goal-Auxiliary Actor-Critic for 6D Robotic Grasping with Point Clouds

arXiv: https://arxiv.org/abs/2010.00824 · HTML: https://mlsanity.com/paper/2010.00824/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Oct 2020.

## Built on by (most cited first, 24 of 24)

- [Scaling Proprioceptive-Visual Learning with Heterogeneous Pre-trained Transformers](https://mlsanity.com/paper/2409.20537/) (2024)
- [DexPoint: Generalizable Point Cloud Reinforcement Learning for Sim-to-Real Dexterous Manipulation](https://mlsanity.com/paper/2211.09423/) (2022)
- [DexArt: Benchmarking Generalizable Dexterous Manipulation with Articulated Objects](https://mlsanity.com/paper/2305.05706/) (2023)
- [NeRF in the Palm of Your Hand: Corrective Augmentation for Robotics via Novel-View Synthesis](https://mlsanity.com/paper/2301.08556/) (2023)
- [GenSim2: Scaling Robot Data Generation with Multi-modal and Reasoning LLMs](https://mlsanity.com/paper/2410.03645/) (2024)
- [PoCo: Policy Composition from and for Heterogeneous Robot Learning](https://mlsanity.com/paper/2402.02511/) (2024)
- [Learning Human-to-Robot Handovers from Point Clouds](https://mlsanity.com/paper/2303.17592/) (2023)
- [HACMan: Learning Hybrid Actor-Critic Maps for 6D Non-Prehensile Manipulation](https://mlsanity.com/paper/2305.03942/) (2023)
- [Learning to Grasp the Ungraspable with Emergent Extrinsic Dexterity](https://mlsanity.com/paper/2211.01500/) (2022)
- [Mean Shift Mask Transformer for Unseen Object Instance Segmentation](https://mlsanity.com/paper/2211.11679/) (2022)
- [XVO: Generalized Visual Odometry via Cross-Modal Self-Training](https://mlsanity.com/paper/2309.16772/) (2023)
- [HandoverSim: A Simulation Framework and Benchmark for Human-to-Robot Object Handovers](https://mlsanity.com/paper/2205.09747/) (2022)
- [GenH2R: Learning Generalizable Human-to-Robot Handover via Scalable Simulation, Demonstration, and Imitation](https://mlsanity.com/paper/2401.00929/) (2024)
- [SynH2R: Synthesizing Hand-Object Motions for Learning Human-to-Robot Handovers](https://mlsanity.com/paper/2311.05599/) (2023)
- [NeuralGrasps: Learning Implicit Representations for Grasps of Multiple Robotic Hands](https://mlsanity.com/paper/2207.02959/) (2022)
- [DA2 Dataset: Toward Dexterity-Aware Dual-Arm Grasping](https://mlsanity.com/paper/2208.00408/) (2022)
- [Robot Fleet Learning via Policy Merging](https://mlsanity.com/paper/2310.01362/) (2023)
- [Neural Motion Fields: Encoding Grasp Trajectories as Implicit Value Functions](https://mlsanity.com/paper/2206.14854/) (2022)
- [Stereo Hand-Object Reconstruction for Human-to-Robot Handover](https://mlsanity.com/paper/2412.07487/) (2024)
- [GraspCoT: Integrating Physical Property Reasoning for 6-DoF Grasping under Flexible Language Instructions](https://mlsanity.com/paper/2503.16013/) (2025)
- [COMBO-Grasp: Learning Constraint-Based Manipulation for Bimanual Occluded Grasping](https://mlsanity.com/paper/2502.08054/) (2025)
- [Visuomotor Grasping with World Models for Surgical Robots](https://mlsanity.com/paper/2508.11200/) (2025)
- [Grasp-MPC: Closed-Loop Visual Grasping via Value-Guided Model Predictive Control](https://mlsanity.com/paper/2509.06201/) (2025)
- [PointNet4D: A Lightweight 4D Point Cloud Video Backbone for Online and Offline Perception in Robotic Applications](https://mlsanity.com/paper/2512.01383/) (2025)

## Abstract (arXiv)

6D robotic grasping beyond top-down bin-picking scenarios is a challenging task. Previous solutions based on 6D grasp synthesis with robot motion planning usually operate in an open-loop setting, which are sensitive to grasp synthesis errors. In this work, we propose a new method for learning closed-loop control policies for 6D grasping. Our policy takes a segmented point cloud of an object from an egocentric camera as input, and outputs continuous 6D control actions of the robot gripper for grasping the object. We combine imitation learning and reinforcement learning and introduce a goal-auxiliary actor-critic algorithm for policy learning. We demonstrate that our learned policy can be integrated into a tabletop 6D grasping system and a human-robot handover system to improve the grasping performance of unseen objects. Our videos and code can be found at https://sites.google.com/view/gaddpg .