# HACMan: Learning Hybrid Actor-Critic Maps for 6D Non-Prehensile Manipulation

arXiv: https://arxiv.org/abs/2305.03942 · HTML: https://mlsanity.com/paper/2305.03942/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 45 references that are in the map.
- First posted on arXiv 6 May 2023.

## Builds on (most cited first, 30 of 37)

- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning](https://mlsanity.com/paper/1910.10897/) (2019)
- [PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes](https://mlsanity.com/paper/1711.00199/) (2017)
- [Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulation](https://mlsanity.com/paper/2209.05451/) (2022)
- [robosuite: A Modular Simulation Framework and Benchmark for Robot Learning](https://mlsanity.com/paper/2009.12293/) (2020)
- [Transporter Networks: Rearranging the Visual World for Robotic Manipulation](https://mlsanity.com/paper/2010.14406/) (2020)
- [Contact-GraspNet: Efficient 6-DoF Grasp Generation in Cluttered Scenes](https://mlsanity.com/paper/2103.14127/) (2021)
- [Neural Descriptor Fields: SE(3)-Equivariant Object Representations for Manipulation](https://mlsanity.com/paper/2112.05124/) (2021)
- [ManiSkill: Generalizable Manipulation Skill Benchmark with Large-Scale Demonstrations](https://mlsanity.com/paper/2107.14483/) (2021)
- [Where2Act: From Pixels to Actions for Articulated 3D Objects](https://mlsanity.com/paper/2101.02692/) (2021)
- [A System for General In-Hand Object Re-Orientation](https://mlsanity.com/paper/2111.03043/) (2021)
- [Learning Synergies between Pushing and Grasping with Self-supervised Deep Reinforcement Learning](https://mlsanity.com/paper/1803.09956/) (2018)
- [Visual Dexterity: In-Hand Reorientation of Novel and Complex Object Shapes](https://mlsanity.com/paper/2211.11744/) (2022)
- [DexPoint: Generalizable Point Cloud Reinforcement Learning for Sim-to-Real Dexterous Manipulation](https://mlsanity.com/paper/2211.09423/) (2022)
- [Deep Reinforcement Learning in Parameterized Action Space](https://mlsanity.com/paper/1511.04143/) (2015)
- [UMPNet: Universal Manipulation Policy Network for Articulated Objects](https://mlsanity.com/paper/2109.05668/) (2021)
- [Parametrized Deep Q-Networks Learning: Reinforcement Learning with Discrete-Continuous Hybrid Action Space](https://mlsanity.com/paper/1810.06394/) (2018)
- [Housekeep: Tidying Virtual Households using Commonsense Reasoning](https://mlsanity.com/paper/2205.10712/) (2022)
- [ToolFlowNet: Robotic Manipulation with Tools via Predicting Tool Flow from Point Clouds](https://mlsanity.com/paper/2211.09006/) (2022)
- [Generalization in Dexterous Manipulation via Geometry-Aware Multi-Task Learning](https://mlsanity.com/paper/2111.03062/) (2021)
- [Divide-and-Conquer Reinforcement Learning](https://mlsanity.com/paper/1711.09874/) (2017)
- [Continuous-Discrete Reinforcement Learning for Hybrid Control in Robotics](https://mlsanity.com/paper/2001.00449/) (2020)
- [Frame Mining: a Free Lunch for Learning Robotic Manipulation from 3D Point Clouds](https://mlsanity.com/paper/2210.07442/) (2022)
- [More than a Million Ways to Be Pushed: A High-Fidelity Experimental Dataset of Planar Pushing](https://mlsanity.com/paper/1604.04038/) (2016)
- [Spatial Action Maps for Mobile Manipulation](https://mlsanity.com/paper/2004.09141/) (2020)

## Built on by (most cited first, 29 of 29)

- [Deep Reinforcement Learning for Robotics: A Survey of Real-World Successes](https://mlsanity.com/paper/2408.03539/) (2024)
- [SplatSim: Zero-Shot Sim2Real Transfer of RGB Manipulation Policies Using Gaussian Splatting](https://mlsanity.com/paper/2409.10161/) (2024)
- [HRP: Human Affordances for Robotic Pre-Training](https://mlsanity.com/paper/2407.18911/) (2024)
- [OK-Robot: What Really Matters in Integrating Open-Knowledge Models for Robotics](https://mlsanity.com/paper/2401.12202/) (2024)
- [DyWA: Dynamics-adaptive World Action Model for Generalizable Non-prehensile Manipulation](https://mlsanity.com/paper/2503.16806/) (2025)
- [PIN-WM: Learning Physics-INformed World Models for Non-Prehensile Manipulation](https://mlsanity.com/paper/2504.16693/) (2025)
- [Render and Diffuse: Aligning Image and Action Spaces for Diffusion-based Behaviour Cloning](https://mlsanity.com/paper/2405.18196/) (2024)
- [On the Efficacy of 3D Point Cloud Reinforcement Learning](https://mlsanity.com/paper/2306.06799/) (2023)
- [InCoRo: In-Context Learning for Robotics Control with Feedback Loops](https://mlsanity.com/paper/2402.05188/) (2024)
- [3D Flow Diffusion Policy: Visuomotor Policy Learning via Generating Flow in 3D Space](https://mlsanity.com/paper/2509.18676/) (2025)
- [CORN: Contact-based Object Representation for Nonprehensile Manipulation of General Unseen Objects](https://mlsanity.com/paper/2403.10760/) (2024)
- [Uncertainty-driven Exploration Strategies for Online Grasp Learning](https://mlsanity.com/paper/2309.12038/) (2023)
- [HACMan++: Spatially-Grounded Motion Primitives for Manipulation](https://mlsanity.com/paper/2407.08585/) (2024)
- [Learning Visuotactile Estimation and Control for Non-prehensile Manipulation under Occlusions](https://mlsanity.com/paper/2412.13157/) (2024)
- [Learning Extrinsic Dexterity with Parameterized Manipulation Primitives](https://mlsanity.com/paper/2310.17785/) (2023)
- [Enhancing Exploration with Diffusion Policies in Hybrid Off-Policy RL: Application to Non-Prehensile Manipulation](https://mlsanity.com/paper/2411.14913/) (2024)
- [CLAMP: Contrastive Learning for 3D Multi-View Action-Conditioned Robotic Manipulation Pretraining](https://mlsanity.com/paper/2602.00937/) (2026)
- [RAPID: Robot Agentic Programming from Demonstrations](https://mlsanity.com/paper/2609.30249/) (2026)
- [Pseudo-Labeling and Contextual Curriculum Learning for Online Grasp Learning in Robotic Bin Picking](https://mlsanity.com/paper/2403.02495/) (2024)
- [A General Control-Theoretic Approach for Reinforcement Learning: Theory and Algorithms](https://mlsanity.com/paper/2406.14753/) (2024)
- [Leveraging Extrinsic Dexterity for Occluded Grasping on Grasp Constraining Walls](https://mlsanity.com/paper/2507.14721/) (2025)
- [A Reinforcement Learning Approach to Non-prehensile Manipulation through Sliding](https://mlsanity.com/paper/2502.17221/) (2025)
- [BEAC: Imitating Complex Exploration and Task-oriented Behaviors for Invisible Object Nonprehensile Manipulation](https://mlsanity.com/paper/2503.16803/) (2025)
- [KEA: Keeping Exploration Alive by Proactively Coordinating Exploration Strategies](https://mlsanity.com/paper/2503.18234/) (2025)
- [ActivePusher: Active Learning and Planning with Residual Physics for Nonprehensile Manipulation](https://mlsanity.com/paper/2506.04646/) (2025)
- [CHDP: Cooperative Hybrid Diffusion Policies for Reinforcement Learning in Parameterized Action Space](https://mlsanity.com/paper/2601.05675/) (2026)
- [Approximately Optimal Global Planning for Contact-Rich SE(2) Manipulation on a Graph of Reachable Sets](https://mlsanity.com/paper/2601.10827/) (2026)
- [Rotation-Aware Point-Cloud Embeddings for Vision-Based In-Hand Reorientation](https://mlsanity.com/paper/2606.21788/) (2026)
- [ZeroBot: Learning from Scratch in Minutes with Generative Real2Sim](https://mlsanity.com/paper/2609.34010/) (2026)

## Abstract (arXiv)

Manipulating objects without grasping them is an essential component of human dexterity, referred to as non-prehensile manipulation. Non-prehensile manipulation may enable more complex interactions with the objects, but also presents challenges in reasoning about gripper-object interactions. In this work, we introduce Hybrid Actor-Critic Maps for Manipulation (HACMan), a reinforcement learning approach for 6D non-prehensile manipulation of objects using point cloud observations. HACMan proposes a temporally-abstracted and spatially-grounded object-centric action representation that consists of selecting a contact location from the object point cloud and a set of motion parameters describing how the robot will move after making contact. We modify an existing off-policy RL algorithm to learn in this hybrid discrete-continuous action representation. We evaluate HACMan on a 6D object pose alignment task in both simulation and in the real world. On the hardest version of our task, with randomized initial poses, randomized 6D goals, and diverse object categories, our policy demonstrates strong generalization to unseen object categories without a performance drop, achieving an 89% success rate on unseen objects in simulation and 50% success rate with zero-shot transfer in the real world. Compared to alternative action representations, HACMan achieves a success rate more than three times higher than the best baseline. With zero-shot sim2real transfer, our policy can successfully manipulate unseen objects in the real world for challenging non-planar goals, using dynamic and contact-rich non-prehensile skills. Videos can be found on the project website: https://hacman-2023.github.io.