# Robot Skill Generalization via Keypoint Integrated Soft Actor-Critic Gaussian Mixture Models

arXiv: https://arxiv.org/abs/2310.15059 · HTML: https://mlsanity.com/paper/2310.15059/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 39 references that are in the map.
- First posted on arXiv 23 Oct 2023.

## Builds on (most cited first, 27 of 27)

- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrations](https://mlsanity.com/paper/1709.10087/) (2017)
- [CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Tasks](https://mlsanity.com/paper/2112.03227/) (2021)
- [BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learning](https://mlsanity.com/paper/2202.02005/) (2022)
- [Residual Reinforcement Learning for Robot Control](https://mlsanity.com/paper/1812.03201/) (2018)
- [Unsupervised Learning for Physical Interaction through Video Prediction](https://mlsanity.com/paper/1605.07157/) (2016)
- [One-Shot Visual Imitation Learning via Meta-Learning](https://mlsanity.com/paper/1709.04905/) (2017)
- [Offline-to-Online Reinforcement Learning via Balanced Replay and Pessimistic Q-Ensemble](https://mlsanity.com/paper/2107.00591/) (2021)
- [How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learned](https://mlsanity.com/paper/2102.02915/) (2021)
- [kPAM: KeyPoint Affordances for Category-Level Robotic Manipulation](https://mlsanity.com/paper/1903.06684/) (2019)
- [One-Shot Imitation from Observing Humans via Domain-Adaptive Meta-Learning](https://mlsanity.com/paper/1802.01557/) (2018)
- [Unseen Object Instance Segmentation for Robotic Environments](https://mlsanity.com/paper/2007.08073/) (2020)
- [Transformers for One-Shot Visual Imitation](https://mlsanity.com/paper/2011.05970/) (2020)
- [Task-Embedded Control Networks for Few-Shot Imitation Learning](https://mlsanity.com/paper/1810.03237/) (2018)
- [Neural Task Graphs: Generalizing to Unseen Tasks from a Single Video Demonstration](https://mlsanity.com/paper/1807.03480/) (2018)
- [SE3-Nets: Learning Rigid Body Motion using Deep Neural Networks](https://mlsanity.com/paper/1606.02378/) (2016)
- [Unsupervised Learning of Visual 3D Keypoints for Control](https://mlsanity.com/paper/2106.07643/) (2021)
- [Watch, Try, Learn: Meta-Learning from Demonstrations and Reward](https://mlsanity.com/paper/1906.03352/) (2019)
- [S3K: Self-Supervised Semantic Keypoints for Robotic Manipulation via Multi-View Consistency](https://mlsanity.com/paper/2009.14711/) (2020)
- [K-VIL: Keypoints-based Visual Imitation Learning](https://mlsanity.com/paper/2209.03277/) (2022)
- [Few-Shot Panoptic Segmentation With Foundation Models](https://mlsanity.com/paper/2309.10726/) (2023)
- [Learning on the Job: Self-Rewarding Offline-to-Online Finetuning for Industrial Insertion of Novel Connectors from Vision](https://mlsanity.com/paper/2210.15206/) (2022)
- [Robot Skill Adaptation via Soft Actor-Critic Gaussian Mixture Models](https://mlsanity.com/paper/2111.13129/) (2021)
- [The Treachery of Images: Bayesian Scene Keypoints for Deep Policy Learning in Robotic Manipulation](https://mlsanity.com/paper/2305.04718/) (2023)
- [Wasserstein Gradient Flows for Optimizing Gaussian Mixture Policies](https://mlsanity.com/paper/2305.10411/) (2023)
- [T3VIP: Transformation-based 3D Video Prediction](https://mlsanity.com/paper/2209.11693/) (2022)

## Built on by (most cited first, 1 of 1)

- [DiWA: Diffusion Policy Adaptation with World Models](https://mlsanity.com/paper/2508.03645/) (2025)

## Abstract (arXiv)

A long-standing challenge for a robotic manipulation system operating in real-world scenarios is adapting and generalizing its acquired motor skills to unseen environments. We tackle this challenge employing hybrid skill models that integrate imitation and reinforcement paradigms, to explore how the learning and adaptation of a skill, along with its core grounding in the scene through a learned keypoint, can facilitate such generalization. To that end, we develop Keypoint Integrated Soft Actor-Critic Gaussian Mixture Models (KIS-GMM) approach that learns to predict the reference of a dynamical system within the scene as a 3D keypoint, leveraging visual observations obtained by the robot's physical interactions during skill learning. Through conducting comprehensive evaluations in both simulated and real-world environments, we show that our method enables a robot to gain a significant zero-shot generalization to novel environments and to refine skills in the target environments faster than learning from scratch. Importantly, this is achieved without the need for new ground truth data. Moreover, our method effectively copes with scene displacements.