# PoCo: Policy Composition from and for Heterogeneous Robot Learning

arXiv: https://arxiv.org/abs/2402.02511 · HTML: https://mlsanity.com/paper/2402.02511/

## Key facts

- Cited by 42 papers in the map (data through 14 Oct 2026).
- Builds on 66 of its 89 references that are in the map.
- First posted on arXiv 4 Feb 2024.

## Builds on (most cited first, 30 of 66)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Diffusion Policy: Visuomotor Policy Learning via Action Diffusion](https://mlsanity.com/paper/2303.04137/) (2023)
- [RT-1: Robotics Transformer for Real-World Control at Scale](https://mlsanity.com/paper/2212.06817/) (2022)
- [Imagen Video: High Definition Video Generation with Diffusion Models](https://mlsanity.com/paper/2210.02303/) (2022)
- [Make-A-Video: Text-to-Video Generation without Text-Video Data](https://mlsanity.com/paper/2209.14792/) (2022)
- [Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware](https://mlsanity.com/paper/2304.13705/) (2023)
- [Ego4D: Around the World in 3,000 Hours of Egocentric Video](https://mlsanity.com/paper/2110.07058/) (2021)
- [Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning](https://mlsanity.com/paper/1910.10897/) (2019)
- [Planning with Diffusion for Flexible Behavior Synthesis](https://mlsanity.com/paper/2205.09991/) (2022)
- [On First-Order Meta-Learning Algorithms](https://mlsanity.com/paper/1803.02999/) (2018)
- [An Introduction to Variational Autoencoders](https://mlsanity.com/paper/1906.02691/) (2019)
- [An Overview of Multi-Task Learning in Deep Neural Networks](https://mlsanity.com/paper/1706.05098/) (2017)
- [R3M: A Universal Visual Representation for Robot Manipulation](https://mlsanity.com/paper/2203.12601/) (2022)
- [Sim-to-Real Transfer of Robotic Control with Dynamics Randomization](https://mlsanity.com/paper/1710.06537/) (2017)
- [Compositional Visual Generation with Composable Diffusion Models](https://mlsanity.com/paper/2206.01714/) (2022)
- [Generative Adversarial Networks: An Overview](https://mlsanity.com/paper/1710.07035/) (2017)
- [Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulation](https://mlsanity.com/paper/2209.05451/) (2022)
- [BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learning](https://mlsanity.com/paper/2202.02005/) (2022)
- [Is Conditional Generative Modeling all you need for Decision-Making?](https://mlsanity.com/paper/2211.15657/) (2022)
- [Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review](https://mlsanity.com/paper/1805.00909/) (2018)

## Built on by (most cited first, 30 of 42)

- [Diffusion Policy Policy Optimization](https://mlsanity.com/paper/2409.00588/) (2024)
- [DexVLA: Vision-Language Model with Plug-In Diffusion Expert for General Robot Control](https://mlsanity.com/paper/2502.05855/) (2025)
- [Steering Your Diffusion Policy with Latent Space Reinforcement Learning](https://mlsanity.com/paper/2506.15799/) (2025)
- [Reconciling Reality through Simulation: A Real-to-Sim-to-Real Approach for Robust Manipulation](https://mlsanity.com/paper/2403.03949/) (2024)
- [InstructVLA: Vision-Language-Action Instruction Tuning from Understanding to Manipulation](https://mlsanity.com/paper/2507.17520/) (2025)
- [Streaming Diffusion Policy: Fast Policy Synthesis with Variable Noise Diffusion Models](https://mlsanity.com/paper/2406.04806/) (2024)
- [GenSim2: Scaling Robot Data Generation with Multi-modal and Reasoning LLMs](https://mlsanity.com/paper/2410.03645/) (2024)
- [Transferable Tactile Transformers for Representation Learning Across Diverse Sensors and Tasks](https://mlsanity.com/paper/2406.13640/) (2024)
- [STRAP: Robot Sub-Trajectory Retrieval for Augmented Policy Learning](https://mlsanity.com/paper/2412.15182/) (2024)
- [Generative Trajectory Stitching through Diffusion Composition](https://mlsanity.com/paper/2503.05153/) (2025)
- [Diffusion Models for Robotic Manipulation: A Survey](https://mlsanity.com/paper/2504.08438/) (2025)
- [Inference-Time Policy Steering through Human Interactions](https://mlsanity.com/paper/2411.16627/) (2024)
- [Deep Generative Models in Robotics: A Survey on Learning from Multimodal Demonstrations](https://mlsanity.com/paper/2408.04380/) (2024)
- [DITTO: Demonstration Imitation by Trajectory Transformation](https://mlsanity.com/paper/2403.15203/) (2024)
- [Skill Expansion and Composition in Parameter Space](https://mlsanity.com/paper/2502.05932/) (2025)
- [PolyTouch: A Robust Multi-Modal Tactile Sensor for Contact-rich Manipulation Using Tactile-Diffusion Policies](https://mlsanity.com/paper/2504.19341/) (2025)
- [A Survey on Imitation Learning for Contact-Rich Tasks in Robotics](https://mlsanity.com/paper/2506.13498/) (2025)
- [Multi-Modal Manipulation via Multi-Modal Policy Consensus](https://mlsanity.com/paper/2509.23468/) (2025)
- [Uncertainty Comes for Free: Human-in-the-Loop Policies with Diffusion Models](https://mlsanity.com/paper/2503.01876/) (2025)
- [CDP: Towards Robust Autoregressive Visuomotor Policy Learning via Causal Diffusion](https://mlsanity.com/paper/2506.14769/) (2025)
- [Modality-Composable Diffusion Policy via Inference-Time Distribution-level Composition](https://mlsanity.com/paper/2503.12466/) (2025)
- [Fine-tuning Diffusion Policies with Backpropagation Through Diffusion Timesteps](https://mlsanity.com/paper/2505.10482/) (2025)
- [Projected Coupled Diffusion for Test-Time Constrained Joint Generation](https://mlsanity.com/paper/2508.10531/) (2025)
- [Multi-Camera View Scaling for Data-Efficient Robot Imitation Learning](https://mlsanity.com/paper/2604.00557/) (2026)
- [Breaking Lock-In: Preserving Steerability under Low-Data VLA Post-Training](https://mlsanity.com/paper/2604.23121/) (2026)
- [Learning to See and Act: Task-Aware Virtual View Exploration for Robotic Manipulation](https://mlsanity.com/paper/2508.05186/) (2025)
- [Flexible Multitask Learning with Factorized Diffusion Policy](https://mlsanity.com/paper/2512.21898/) (2025)
- [LodeStar: Long-horizon Dexterity via Synthetic Data Augmentation from Human Demonstrations](https://mlsanity.com/paper/2508.17547/) (2025)
- [Composing Diffusion Policies for Few-shot Learning of Movement Trajectories](https://mlsanity.com/paper/2410.17479/) (2024)
- [The Unreasonable Effectiveness of Discrete-Time Gaussian Process Mixtures for Robot Policy Learning](https://mlsanity.com/paper/2505.03296/) (2025)

## Abstract (arXiv)

Training general robotic policies from heterogeneous data for different tasks is a significant challenge. Existing robotic datasets vary in different modalities such as color, depth, tactile, and proprioceptive information, and collected in different domains such as simulation, real robots, and human videos. Current methods usually collect and pool all data from one domain to train a single policy to handle such heterogeneity in tasks and domains, which is prohibitively expensive and difficult. In this work, we present a flexible approach, dubbed Policy Composition, to combine information across such diverse modalities and domains for learning scene-level and task-level generalized manipulation skills, by composing different data distributions represented with diffusion models. Our method can use task-level composition for multi-task manipulation and be composed with analytic cost functions to adapt policy behaviors at inference time. We train our method on simulation, human, and real robot data and evaluate in tool-use tasks. The composed policy achieves robust and dexterous performance under varying scenes and tasks and outperforms baselines from a single data source in both simulation and real-world experiments. See https://liruiw.github.io/policycomp for more details .