# Where Small Mistakes Matter: Quantifying State Criticality for Imitation Learning

arXiv: https://arxiv.org/abs/2610.17709 · HTML: https://mlsanity.com/paper/2610.17709/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 29 of its 31 references that are in the map.
- First posted on arXiv 13 Oct 2026.

## Placed in

- Offline RL and imitation learning from data / [Imitation learning methods](https://mlsanity.com/offlinerl/imitation-learning-methods/)

## Builds on (most cited first, 29 of 29)

- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Diffusion Policy: Visuomotor Policy Learning via Action Diffusion](https://mlsanity.com/paper/2303.04137/) (2023)
- [π0.5: a Vision-Language-Action Model with Open-World Generalization](https://mlsanity.com/paper/2504.16054/) (2025)
- [Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware](https://mlsanity.com/paper/2304.13705/) (2023)
- [Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learning](https://mlsanity.com/paper/1910.10897/) (2019)
- [DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset](https://mlsanity.com/paper/2403.12945/) (2024)
- [ManiSkill2: A Unified Benchmark for Generalizable Manipulation Skills](https://mlsanity.com/paper/2302.04659/) (2023)
- [Behavior Transformers: Cloning k modes with one stone](https://mlsanity.com/paper/2206.11251/) (2022)
- [Residual Reinforcement Learning for Robot Control](https://mlsanity.com/paper/1812.03201/) (2018)
- [From Imitation to Refinement -- Residual RL for Precise Assembly](https://mlsanity.com/paper/2407.16677/) (2024)
- [Self-Improving Vision-Language-Action Models with Data Generation via Residual RL](https://mlsanity.com/paper/2511.00091/) (2025)
- [Data Quality in Imitation Learning](https://mlsanity.com/paper/2306.02437/) (2023)
- [Policy Decorator: Model-Agnostic Online Refinement for Large Policy Model](https://mlsanity.com/paper/2412.13630/) (2024)
- [Residual Off-Policy RL for Finetuning Behavior Cloning Policies](https://mlsanity.com/paper/2509.19301/) (2025)
- [Establishing Appropriate Trust via Critical States](https://mlsanity.com/paper/1810.08174/) (2018)
- [Re-Mix: Optimizing Data Mixtures for Large Scale Imitation Learning](https://mlsanity.com/paper/2408.14037/) (2024)
- [Robot Data Curation with Mutual Information Estimators](https://mlsanity.com/paper/2502.08623/) (2025)
- [DataMIL: Selecting Data for Robot Imitation Learning with Datamodels](https://mlsanity.com/paper/2505.09603/) (2025)
- [CUPID: Curating Data your Robot Loves with Influence Functions](https://mlsanity.com/paper/2506.19121/) (2025)
- [Learning Contracting Vector Fields For Stable Imitation Learning](https://mlsanity.com/paper/1804.04878/) (2018)
- [Curating Demonstrations using Online Experience](https://mlsanity.com/paper/2503.03707/) (2025)
- [Learning to Discern: Imitating Heterogeneous Human Demonstrations with Preference and Representation Learning](https://mlsanity.com/paper/2310.14196/) (2023)
- [Stable-BC: Controlling Covariate Shift with Stable Behavior Cloning](https://mlsanity.com/paper/2408.06246/) (2024)
- [SAFE-GIL: SAFEty Guided Imitation Learning for Robotic Systems](https://mlsanity.com/paper/2404.05249/) (2024)
- [Visual Backtracking Teleoperation: A Data Collection Protocol for Offline Image-Based Reinforcement Learning](https://mlsanity.com/paper/2210.02343/) (2022)
- [Surprise Potential as a Measure of Interactivity in Driving Scenarios](https://mlsanity.com/paper/2502.05677/) (2025)
- [Towards Balanced Behavior Cloning from Imbalanced Datasets](https://mlsanity.com/paper/2508.06319/) (2025)
- [Identifying Critical States by the Action-Based Variance of Expected Return](https://mlsanity.com/paper/2008.11332/) (2020)
- [Safety-Aware Imitation Learning via MPC-Guided Disturbance Injection](https://mlsanity.com/paper/2508.03129/) (2025)

## Abstract (arXiv)

Imitation learning typically treats every demonstration state as equally important, optimizing a behavior-cloning objective uniformly across the dataset. Yet not all states are created equal: in a small number of critical states, even a slight deviation from the expert action can lead to a failure, while in many other states such errors are recoverable and inconsequential. We formalize state criticality in imitation learning as the worst-case degradation in downstream value induced by a bounded perturbation of the expert action at a given state. We then propose SCRIM, a practical State CRIticality Measure that estimates this quantity by training an adversarial disturbance policy and a value critic around a fixed imitation policy. SCRIM assigns each demonstration state a score reflecting how vulnerable the task outcome is to small, policy-relevant action errors. We show that these scores can be used to reweight behavior-cloning losses, improving imitation at consequential states and increasing overall task performance. We further show that SCRIM enables data curation by selecting compact, high-value subsets of demonstrations that preserve performance while substantially reducing the amount of training data. We evaluate SCRIM across various simulated and real-world robotic manipulation tasks, and demonstrate that SCRIM improves policy training, enables efficient data selection, and provides interpretable insights into which moments in a demonstration are most critical for task success. Project website: https://sia-lab-git.github.io/SCRIM/