# Point Policy: Unifying Observations and Actions with Key Points for Robot Manipulation

arXiv: https://arxiv.org/abs/2502.20391 · HTML: https://mlsanity.com/paper/2502.20391/

## Key facts

- Cited by 47 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Feb 2025.

## Built on by (most cited first, 30 of 47)

- [PointWorld: Scaling 3D World Models for In-The-Wild Robotic Manipulation](https://mlsanity.com/paper/2601.03782/) (2026)
- [EgoZero: Robot Learning from Smart Glasses](https://mlsanity.com/paper/2505.20290/) (2025)
- [ZeroMimic: Distilling Robotic Manipulation Skills from Web Videos](https://mlsanity.com/paper/2503.23877/) (2025)
- [Dream2Flow: Bridging Video Generation and Open-World Manipulation with 3D Object Flow](https://mlsanity.com/paper/2512.24766/) (2025)
- [EgoWAM: World Action Models Beyond Pixels with In-the-Wild Egocentric Human Data](https://mlsanity.com/paper/2607.08436/) (2026)
- [Dexterity from Smart Lenses: Multi-Fingered Robot Manipulation with In-the-Wild Human Demonstrations](https://mlsanity.com/paper/2511.16661/) (2025)
- [AMPLIFY: Actionless Motion Priors for Robot Learning from Videos](https://mlsanity.com/paper/2506.14198/) (2025)
- [HumanEgo: Zero-Shot Robot Learning from Minutes of Human Egocentric Videos](https://mlsanity.com/paper/2605.24934/) (2026)
- [MaskWAM: Unifying Mask Prompting and Prediction for World-Action Models](https://mlsanity.com/paper/2606.13515/) (2026)
- [Touch begins where vision ends: Generalizable policies for contact-rich manipulation](https://mlsanity.com/paper/2506.13762/) (2025)
- [GP3: A 3D Geometry-Aware Policy with Multi-View Images for Robotic Manipulation](https://mlsanity.com/paper/2509.15733/) (2025)
- [GuidedVLA: Specifying Task-Relevant Factors via Plug-and-Play Action Attention Specialization](https://mlsanity.com/paper/2605.12369/) (2026)
- [Scaffolding Dexterous Manipulation with Vision-Language Models](https://mlsanity.com/paper/2506.19212/) (2025)
- [H2R-Grounder: A Paired-Data-Free Paradigm for Translating Human Interaction Videos into Physically Grounded Robot Videos](https://mlsanity.com/paper/2512.09406/) (2025)
- [Feel the Force: Contact-Driven Learning from Humans](https://mlsanity.com/paper/2506.01944/) (2025)
- [X-Diffusion: Training Diffusion Policies on Cross-Embodiment Human Demonstrations](https://mlsanity.com/paper/2511.04671/) (2025)
- [Robot Learning from Human Videos: A Survey](https://mlsanity.com/paper/2604.27621/) (2026)
- [LUCID: Learning Embodiment-Agnostic Intent Models from Unstructured Human Videos for Scalable Dexterous Robot Skill Acquisition](https://mlsanity.com/paper/2606.11628/) (2026)
- [Human Universal Grasping](https://mlsanity.com/paper/2606.17054/) (2026)
- [LodeStar: Long-horizon Dexterity via Synthetic Data Augmentation from Human Demonstrations](https://mlsanity.com/paper/2508.17547/) (2025)
- [Contact-Anchored Policies: Contact Conditioning Creates Strong Robot Utility Models](https://mlsanity.com/paper/2602.09017/) (2026)
- [EasyMimic: A Low-Cost Framework for Robot Imitation Learning from Human Videos](https://mlsanity.com/paper/2602.11464/) (2026)
- [Tether: Autonomous Functional Play with Correspondence-Driven Trajectory Warping](https://mlsanity.com/paper/2603.03278/) (2026)
- [On the Generalization Capabilities, Design Choices and Limitations of Keypoint Imitation Learning](https://mlsanity.com/paper/2605.26649/) (2026)
- [GHOST: Hierarchical Sub-Goal Policies for Generalizing Robot Manipulation](https://mlsanity.com/paper/2606.10025/) (2026)
- [Dexterous Point Policy: Learning Point-based Dexterous Hand Policies from Human Demonstrations](https://mlsanity.com/paper/2606.10614/) (2026)
- [ToolTipNet: A Segmentation-Driven Deep Learning Baseline for Surgical Instrument Tip Detection](https://mlsanity.com/paper/2504.09700/) (2025)
- [EgoLoc: A Generalizable Solution for Temporal Interaction Localization in Egocentric Videos](https://mlsanity.com/paper/2508.12349/) (2025)
- [Correspondence-Oriented Imitation Learning: Flexible Visuomotor Control with 3D Conditioning](https://mlsanity.com/paper/2512.05953/) (2025)
- [Atomic Action Slicing: Planner-Aligned Options for Generalist VLA Agents](https://mlsanity.com/paper/2512.11584/) (2025)

## Abstract (arXiv)

Building robotic agents capable of operating across diverse environments and object types remains a significant challenge, often requiring extensive data collection. This is particularly restrictive in robotics, where each data point must be physically executed in the real world. Consequently, there is a critical need for alternative data sources for robotics and frameworks that enable learning from such data. In this work, we present Point Policy, a new method for learning robot policies exclusively from offline human demonstration videos and without any teleoperation data. Point Policy leverages state-of-the-art vision models and policy architectures to translate human hand poses into robot poses while capturing object states through semantically meaningful key points. This approach yields a morphology-agnostic representation that facilitates effective policy learning. Our experiments on 8 real-world tasks demonstrate an overall 75% absolute improvement over prior works when evaluated in identical settings as training. Further, Point Policy exhibits a 74% gain across tasks for novel object instances and is robust to significant background clutter. Videos of the robot are best viewed at https://point-policy.github.io/.