# Hybrid Imitation Learning: Teleoperation Augmentation Primitives that Policies Learn to Trigger

arXiv: https://arxiv.org/abs/2512.04960 · HTML: https://mlsanity.com/paper/2512.04960/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Robot manipulation imitation learning, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 30 of its 45 references that are in the map.
- First posted on arXiv 4 Dec 2025.

## Placed in

- Offline RL and imitation learning from data / [Robot manipulation imitation learning](https://mlsanity.com/offlinerl/robot-manipulation-imitation-learning/)

## Builds on (most cited first, 30 of 30)

- [Diffusion Policy: Visuomotor Policy Learning via Action Diffusion](https://mlsanity.com/paper/2303.04137/) (2023)
- [Do As I Can, Not As I Say: Grounding Language in Robotic Affordances](https://mlsanity.com/paper/2204.01691/) (2022)
- [Code as Policies: Language Model Programs for Embodied Control](https://mlsanity.com/paper/2209.07753/) (2022)
- [DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset](https://mlsanity.com/paper/2403.12945/) (2024)
- [What Matters in Learning from Offline Human Demonstrations for Robot Manipulation](https://mlsanity.com/paper/2108.03298/) (2021)
- [robosuite: A Modular Simulation Framework and Benchmark for Robot Learning](https://mlsanity.com/paper/2009.12293/) (2020)
- [MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrations](https://mlsanity.com/paper/2310.17596/) (2023)
- [HG-DAgger: Interactive Imitation Learning with Human Experts](https://mlsanity.com/paper/1810.02890/) (2018)
- [OmniH2O: Universal and Dexterous Human-to-Humanoid Whole-Body Teleoperation and Learning](https://mlsanity.com/paper/2406.08858/) (2024)
- [A Careful Examination of Large Behavior Models for Multitask Dexterous Manipulation](https://mlsanity.com/paper/2507.05331/) (2025)
- [Human-in-the-Loop Imitation Learning using Remote Teleoperation](https://mlsanity.com/paper/2012.06733/) (2020)
- [Robot Learning on the Job: Human-in-the-Loop Autonomy and Learning During Deployment](https://mlsanity.com/paper/2211.08416/) (2022)
- [TWIST: Teleoperated Whole-Body Imitation System](https://mlsanity.com/paper/2505.02833/) (2025)
- [EnsembleDAgger: A Bayesian Approach to Safe Imitation Learning](https://mlsanity.com/paper/1807.08364/) (2018)
- [Augmenting Reinforcement Learning with Behavior Primitives for Diverse Manipulation Tasks](https://mlsanity.com/paper/2110.03655/) (2021)
- [SkillDiffuser: Interpretable Hierarchical Planning via Skill Abstractions in Diffusion-Based Task Execution](https://mlsanity.com/paper/2312.11598/) (2023)
- [Waypoint-Based Imitation Learning for Robotic Manipulation](https://mlsanity.com/paper/2307.14326/) (2023)
- [HYDRA: Hybrid Robot Actions for Imitation Learning](https://mlsanity.com/paper/2306.17237/) (2023)
- [AirExo: Low-Cost Exoskeletons for Learning Whole-Arm Manipulation in the Wild](https://mlsanity.com/paper/2309.14975/) (2023)
- [BEHAVIOR Robot Suite: Streamlining Real-World Whole-Body Manipulation for Everyday Household Activities](https://mlsanity.com/paper/2503.05652/) (2025)
- [What's the Move? Hybrid Imitation Learning via Salient Points](https://mlsanity.com/paper/2412.05426/) (2024)
- [PRIME: Scaffolding Manipulation Tasks with Behavior Primitives for Data-Efficient Imitation Learning](https://mlsanity.com/paper/2403.00929/) (2024)
- [A Robotic Skill Learning System Built Upon Diffusion Policies and Foundation Models](https://mlsanity.com/paper/2403.16730/) (2024)
- [HoMeR: Learning In-the-Wild Mobile Manipulation via Hybrid Imitation and Whole-Body Control](https://mlsanity.com/paper/2506.01185/) (2025)
- [A Virtual Reality Framework for Human-Robot Collaboration in Cloth Folding](https://mlsanity.com/paper/2305.07493/) (2023)
- [Real-Time Operator Takeover for Visuomotor Diffusion Policy Training](https://mlsanity.com/paper/2502.02308/) (2025)
- [Puppeteer Your Robot: Augmented Reality Leader-Follower Teleoperation](https://mlsanity.com/paper/2407.11741/) (2024)
- [LLM-Driven Augmented Reality Puppeteer: Controller-Free Voice-Commanded Robot Teleoperation](https://mlsanity.com/paper/2502.09142/) (2025)
- [Shared-Control Robotic Manipulation in Virtual Reality](https://mlsanity.com/paper/2205.10564/) (2022)
- [Gesture First, LLM-Assisted Voice Complement: Exploring Multimodal Robot 'Puppeteer' Teleoperation Via Virtual Counterpart in Augmented Reality](https://mlsanity.com/paper/2506.13189/) (2025)

## Abstract (arXiv)

What an operator can demonstrate bounds what imitation learning can learn. Teleoperation interfaces map the human body to the robot, so motions that are hard for a human, such as holding an exact orientation, returning to the same viewpoint, or turning a wrist joint several full revolutions, are difficult to demonstrate on most teleoperation interfaces, even when they are trivial for the robot. We introduce Teleoperation Augmentation Primitives (TAPs): axis locks, perching waypoints, pose anchors, and embodiment-specific routines that the operator triggers during a demonstration by speech, an AR menu, or simply with a controller button. TAPs are themselves recorded in the demonstration and can therefore also be learned and invoked by the policy itself. In simulation, where benchmarks provide human demonstrations, we show that augmenting them with TAPs after the fact yields better policies on a wristcamera-only peg insertion (0.372 to 0.531 success) and on a mug-cleanup task with a learned "remember this pose" anchor (0.203 to 0.323). On a real robot, three tasks show the same pattern: primitives that help the operator do not hurt the policy, and a policy triggering an unscrewing routine succeeds where a plain policy cannot resolve the multi-turn motion from images alone (67% vs. 38% success). We close with an industrial proof-of-concept case leveraging this idea. Project website: https://hybrid-imitation.github.io/