# Unsupervised Learning of Visual 3D Keypoints for Control

arXiv: https://arxiv.org/abs/2106.07643 · HTML: https://mlsanity.com/paper/2106.07643/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 42 references that are in the map.
- First posted on arXiv 14 Jun 2021.

## Builds on (most cited first, 30 of 30)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [WaveNet: A Generative Model for Raw Audio](https://mlsanity.com/paper/1609.03499/) (2016)
- [Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning](https://mlsanity.com/paper/1910.10897/) (2019)
- [Curiosity-driven Exploration by Self-supervised Prediction](https://mlsanity.com/paper/1705.05363/) (2017)
- [CURL: Contrastive Unsupervised Representations for Reinforcement Learning](https://mlsanity.com/paper/2004.04136/) (2020)
- [First Order Motion Model for Image Animation](https://mlsanity.com/paper/2003.00196/) (2020)
- [Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixels](https://mlsanity.com/paper/2004.13649/) (2020)
- [Reinforcement Learning with Augmented Data](https://mlsanity.com/paper/2004.14990/) (2020)
- [Reinforcement Learning with Unsupervised Auxiliary Tasks](https://mlsanity.com/paper/1611.05397/) (2016)
- [Large-Scale Study of Curiosity-Driven Learning](https://mlsanity.com/paper/1808.04355/) (2018)
- [Animating Arbitrary Objects via Deep Motion Transfer](https://mlsanity.com/paper/1812.08861/) (2018)
- [kPAM: KeyPoint Affordances for Category-Level Robotic Manipulation](https://mlsanity.com/paper/1903.06684/) (2019)
- [Dense Object Nets: Learning Dense Visual Object Descriptors By and For Robotic Manipulation](https://mlsanity.com/paper/1806.08756/) (2018)
- [Unsupervised Learning of Object Landmarks through Conditional Image Generation](https://mlsanity.com/paper/1806.07823/) (2018)
- [Unsupervised Learning of Object Keypoints for Perception and Control](https://mlsanity.com/paper/1906.11883/) (2019)
- [Unsupervised Learning of Object Structure and Dynamics from Videos](https://mlsanity.com/paper/1906.07889/) (2019)
- [Discovery of Latent 3D Keypoints via End-to-end Geometric Reasoning](https://mlsanity.com/paper/1807.03146/) (2018)
- [DeepSFM: Structure From Motion Via Deep Bundle Adjustment](https://mlsanity.com/paper/1912.09697/) (2019)
- [Keypoints into the Future: Self-Supervised Correspondence in Model-Based Reinforcement Learning](https://mlsanity.com/paper/2009.05085/) (2020)
- [Learning Rope Manipulation Policies Using Dense Object Descriptors Trained on Synthetic Depth Data](https://mlsanity.com/paper/2003.01835/) (2020)
- [Assistive Gym: A Physics Simulation Framework for Assistive Robotics](https://mlsanity.com/paper/1910.04700/) (2019)
- [Unsupervised Keypoint Learning for Guiding Class-Conditional Video Prediction](https://mlsanity.com/paper/1910.02027/) (2019)
- [kPAM-SC: Generalizable Manipulation Planning using KeyPoint Affordance and Shape Completion](https://mlsanity.com/paper/1909.06980/) (2019)
- [Self-Supervised 3D Keypoint Learning for Ego-motion Estimation](https://mlsanity.com/paper/1912.03426/) (2019)
- [Learning Human Behaviors for Robot-Assisted Dressing](https://mlsanity.com/paper/1709.07033/) (2017)

## Built on by (most cited first, 30 of 30)

- [ReKep: Spatio-Temporal Reasoning of Relational Keypoint Constraints for Robotic Manipulation](https://mlsanity.com/paper/2409.01652/) (2024)
- [Human-to-Robot Imitation in the Wild](https://mlsanity.com/paper/2207.09450/) (2022)
- [Multi-View Masked World Models for Visual Robotic Manipulation](https://mlsanity.com/paper/2302.02408/) (2023)
- [Look Closer: Bridging Egocentric and Third-Person Views with Transformers for Robotic Manipulation](https://mlsanity.com/paper/2201.07779/) (2022)
- [Visual Reinforcement Learning with Self-Supervised 3D Representations](https://mlsanity.com/paper/2210.07241/) (2022)
- [DynaMo: In-Domain Dynamics Pretraining for Visuo-Motor Control](https://mlsanity.com/paper/2409.12192/) (2024)
- [View-Invariant Policy Learning via Zero-Shot Novel View Synthesis](https://mlsanity.com/paper/2409.03685/) (2024)
- [Frame Mining: a Free Lunch for Learning Robotic Manipulation from 3D Point Clouds](https://mlsanity.com/paper/2210.07442/) (2022)
- [USEEK: Unsupervised SE(3)-Equivariant 3D Keypoints for Generalizable Manipulation](https://mlsanity.com/paper/2209.13864/) (2022)
- [UAD: Unsupervised Affordance Distillation for Generalization in Robotic Manipulation](https://mlsanity.com/paper/2506.09284/) (2025)
- [3D Implicit Transporter for Temporally Consistent Keypoint Discovery](https://mlsanity.com/paper/2309.05098/) (2023)
- [Back to 3D: Few-Shot 3D Keypoint Detection with Back-Projected 2D Features](https://mlsanity.com/paper/2311.18113/) (2023)
- [MoVie: Visual Model-Based Policy Adaptation for View Generalization](https://mlsanity.com/paper/2307.00972/) (2023)
- [The Treachery of Images: Bayesian Scene Keypoints for Deep Policy Learning in Robotic Manipulation](https://mlsanity.com/paper/2305.04718/) (2023)
- [BKinD-3D: Self-Supervised 3D Keypoint Discovery from Multi-View Videos](https://mlsanity.com/paper/2212.07401/) (2022)
- [Tool-as-Interface: Learning Robot Policies from Observing Human Tool Use](https://mlsanity.com/paper/2504.04612/) (2025)
- [Multi-View Dreaming: Multi-View World Model with Contrastive Learning](https://mlsanity.com/paper/2203.11024/) (2022)
- [Visual Spatial Attention and Proprioceptive Data-Driven Reinforcement Learning for Robust Peg-in-Hole Task Under Variable Conditions](https://mlsanity.com/paper/2312.16438/) (2023)
- [Learning Keypoints from Synthetic Data for Robotic Cloth Folding](https://mlsanity.com/paper/2205.06714/) (2022)
- [Self-Supervised Learning of Multi-Object Keypoints for Robotic Manipulation](https://mlsanity.com/paper/2205.08316/) (2022)
- [Towards Learning Geometric Eigen-Lengths Crucial for Fitting Tasks](https://mlsanity.com/paper/2312.15610/) (2023)
- [Neural Marionette: Unsupervised Learning of Motion Skeleton and Latent Dynamics from Volumetric Video](https://mlsanity.com/paper/2202.08418/) (2022)
- [Unsupervised 3D Keypoint Discovery with Multi-View Geometry](https://mlsanity.com/paper/2211.12829/) (2022)
- [On the Generalization Capabilities, Design Choices and Limitations of Keypoint Imitation Learning](https://mlsanity.com/paper/2605.26649/) (2026)
- [KINet: Unsupervised Forward Models for Robotic Pushing Manipulation](https://mlsanity.com/paper/2202.09006/) (2022)
- [Robot Skill Generalization via Keypoint Integrated Soft Actor-Critic Gaussian Mixture Models](https://mlsanity.com/paper/2310.15059/) (2023)
- [SelfGeo: Self-supervised and Geodesic-consistent Estimation of Keypoints on Deformable Shapes](https://mlsanity.com/paper/2408.02291/) (2024)
- [Learning Fused State Representations for Control from Multi-View Observations](https://mlsanity.com/paper/2502.01316/) (2025)
- [KeyCLD: Learning Constrained Lagrangian Dynamics in Keypoint Coordinates from Images](https://mlsanity.com/paper/2206.11030/) (2022)
- [From Causal Graphs to Markovian States: Multi-Order State Exposure for Deep RL](https://mlsanity.com/paper/2605.07057/) (2026)

## Abstract (arXiv)

Learning sensorimotor control policies from high-dimensional images crucially relies on the quality of the underlying visual representations. Prior works show that structured latent space such as visual keypoints often outperforms unstructured representations for robotic control. However, most of these representations, whether structured or unstructured are learned in a 2D space even though the control tasks are usually performed in a 3D environment. In this work, we propose a framework to learn such a 3D geometric structure directly from images in an end-to-end unsupervised manner. The input images are embedded into latent 3D keypoints via a differentiable encoder which is trained to optimize both a multi-view consistency loss and downstream task objective. These discovered 3D keypoints tend to meaningfully capture robot joints as well as object movements in a consistent manner across both time and 3D space. The proposed approach outperforms prior state-of-art methods across a variety of reinforcement learning benchmarks. Code and videos at https://buoyancy99.github.io/unsup-3d-keypoints/