# SkiP: When to Skip and When to Refine for Efficient Robot Manipulation

arXiv: https://arxiv.org/abs/2605.15536 · HTML: https://mlsanity.com/paper/2605.15536/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Robot manipulation imitation learning, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 28 of its 35 references that are in the map.
- First posted on arXiv 15 May 2026.

## Placed in

- Offline RL and imitation learning from data / [Robot manipulation imitation learning](https://mlsanity.com/offlinerl/robot-manipulation-imitation-learning/)

## Builds on (most cited first, 28 of 28)

- [RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control](https://mlsanity.com/paper/2307.15818/) (2023)
- [Diffusion Policy: Visuomotor Policy Learning via Action Diffusion](https://mlsanity.com/paper/2303.04137/) (2023)
- [RT-1: Robotics Transformer for Real-World Control at Scale](https://mlsanity.com/paper/2212.06817/) (2022)
- [π0.5: a Vision-Language-Action Model with Open-World Generalization](https://mlsanity.com/paper/2504.16054/) (2025)
- [Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware](https://mlsanity.com/paper/2304.13705/) (2023)
- [What Matters in Learning from Offline Human Demonstrations for Robot Manipulation](https://mlsanity.com/paper/2108.03298/) (2021)
- [RLBench: The Robot Learning Benchmark & Learning Environment](https://mlsanity.com/paper/1909.12271/) (2019)
- [FAST: Efficient Action Tokenization for Vision-Language-Action Models](https://mlsanity.com/paper/2501.09747/) (2025)
- [CLIPort: What and Where Pathways for Robotic Manipulation](https://mlsanity.com/paper/2109.12098/) (2021)
- [CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Tasks](https://mlsanity.com/paper/2112.03227/) (2021)
- [Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulation](https://mlsanity.com/paper/2209.05451/) (2022)
- [Behavior Transformers: Cloning k modes with one stone](https://mlsanity.com/paper/2206.11251/) (2022)
- [3D Diffuser Actor: Policy Diffusion with 3D Scene Representations](https://mlsanity.com/paper/2402.10885/) (2024)
- [RVT: Robotic View Transformer for 3D Object Manipulation](https://mlsanity.com/paper/2306.14896/) (2023)
- [Act3D: 3D Feature Field Transformers for Multi-Task Robotic Manipulation](https://mlsanity.com/paper/2306.17817/) (2023)
- [RoboTwin: Dual-Arm Robot Benchmark with Generative Digital Twins](https://mlsanity.com/paper/2504.13059/) (2025)
- [Instruction-driven history-aware policies for robotic manipulations](https://mlsanity.com/paper/2209.04899/) (2022)
- [Autoregressive Action Sequence Learning for Robotic Manipulation](https://mlsanity.com/paper/2410.03132/) (2024)
- [Keyframe-Focused Visual Imitation Learning](https://mlsanity.com/paper/2106.06452/) (2021)
- [SAIL: Faster-than-Demonstration Execution of Imitation Learning Policies](https://mlsanity.com/paper/2506.11948/) (2025)
- [Chain-of-Action: Trajectory Autoregressive Modeling for Robotic Manipulation](https://mlsanity.com/paper/2506.09990/) (2025)
- [RoboTron-Mani: All-in-One Multimodal Large Model for Robotic Manipulation](https://mlsanity.com/paper/2412.07215/) (2024)
- [Coarse-to-Fine Q-attention with Learned Path Ranking](https://mlsanity.com/paper/2204.01571/) (2022)
- [DaDu-Corki: Algorithm-Architecture Co-Design for Embodied AI-powered Robotic Manipulation](https://mlsanity.com/paper/2407.04292/) (2024)
- [FreqPolicy: Efficient Flow-based Visuomotor Policy via Frequency Consistency](https://mlsanity.com/paper/2506.08822/) (2025)
- [FreqPolicy: Frequency Autoregressive Visuomotor Policy with Continuous Tokens](https://mlsanity.com/paper/2506.01583/) (2025)
- [RoboCAS: A Benchmark for Robotic Manipulation in Complex Object Arrangement Scenarios](https://mlsanity.com/paper/2407.06951/) (2024)
- [Learning to See and Act: Task-Aware Virtual View Exploration for Robotic Manipulation](https://mlsanity.com/paper/2508.05186/) (2025)

## Abstract (arXiv)

Previous imitation learning policies predict future actions at every control step, whether in smooth motion phases or precise, contact-rich operation phases. This uniform treatment is wasteful: most steps in a manipulation trajectory traverse free space and carry little task-relevant information, while a small fraction of key steps around contacts, grasps, and alignment demand dense, high-resolution prediction. We propose a novel action relabeling mechanism: at each timestep in a skip segment, we replace the behavior cloning target with the action at the entrance of the next key segment, enabling the policy to leap over redundant steps in a single decision. The resulting Skip Policy (SkiP) dynamically leaps over skip segments and intensively refines actions in key segments, within a single unified network requiring no learned skip planner or hierarchical structure. To automatically partition demonstrations into key and skip segments without manual annotation, we introduce Motion Spectrum Keying (MSK), a fast, task-agnostic procedure that detects local motion complexity from action signals. Extensive experiments across 72 simulated manipulation tasks and three real-robot tasks show that SkiP reduces executed steps by 15--40% while matching or improving success rates across various policy backbones. Project page: texttthttps://pgq18.github.io/SkiP-page/.