# Curating Demonstrations using Online Experience

arXiv: https://arxiv.org/abs/2503.03707 · HTML: https://mlsanity.com/paper/2503.03707/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 49 references that are in the map.
- First posted on arXiv 5 Mar 2025.

## Builds on (most cited first, 30 of 33)

- [Diffusion Policy: Visuomotor Policy Learning via Action Diffusion](https://mlsanity.com/paper/2303.04137/) (2023)
- [π0: A Vision-Language-Action Flow Model for General Robot Control](https://mlsanity.com/paper/2410.24164/) (2024)
- [Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware](https://mlsanity.com/paper/2304.13705/) (2023)
- [DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset](https://mlsanity.com/paper/2403.12945/) (2024)
- [What Matters in Learning from Offline Human Demonstrations for Robot Manipulation](https://mlsanity.com/paper/2108.03298/) (2021)
- [Deduplicating Training Data Makes Language Models Better](https://mlsanity.com/paper/2107.06499/) (2021)
- [DoReMi: Optimizing Data Mixtures Speeds Up Language Model Pretraining](https://mlsanity.com/paper/2305.10429/) (2023)
- [Extrapolating Beyond Suboptimal Demonstrations via Inverse Reinforcement Learning from Observations](https://mlsanity.com/paper/1904.06387/) (2019)
- [RoboAgent: Generalization and Efficiency in Robot Manipulation via Semantic Augmentations and Action Chunking](https://mlsanity.com/paper/2309.01918/) (2023)
- [Behavior Generation with Latent Actions](https://mlsanity.com/paper/2403.03181/) (2024)
- [A Survey of Imitation Learning: Algorithms, Recent Developments, and Challenges](https://mlsanity.com/paper/2309.02473/) (2023)
- [Reinforcement Learning Upside Down: Don't Predict Rewards -- Just Map Them to Actions](https://mlsanity.com/paper/1912.02875/) (2019)
- [3D Diffusion Policy: Generalizable Visuomotor Policy Learning via Simple 3D Representations](https://mlsanity.com/paper/2403.03954/) (2024)
- [Imitation Learning from Imperfect Demonstration](https://mlsanity.com/paper/1901.09387/) (2019)
- [Reward-Conditioned Policies](https://mlsanity.com/paper/1912.13465/) (2019)
- [Learning Latent Representations to Influence Multi-Agent Interaction](https://mlsanity.com/paper/2011.06619/) (2020)
- [Discriminator-Weighted Offline Imitation Learning from Suboptimal Demonstrations](https://mlsanity.com/paper/2207.10050/) (2022)
- [Uncertainty-Aware Reinforcement Learning for Collision Avoidance](https://mlsanity.com/paper/1702.01182/) (2017)
- [AutoRT: Embodied Foundation Models for Large Scale Orchestration of Robotic Agents](https://mlsanity.com/paper/2401.12963/) (2024)
- [BAKU: An Efficient Transformer for Multi-Task Policy Learning](https://mlsanity.com/paper/2406.07539/) (2024)
- [Learning and Retrieval from Prior Data for Skill-based Imitation Learning](https://mlsanity.com/paper/2210.11435/) (2022)
- [Data Quality in Imitation Learning](https://mlsanity.com/paper/2306.02437/) (2023)
- [Equivariant Diffusion Policy](https://mlsanity.com/paper/2407.01812/) (2024)
- [Behavior Retrieval: Few-Shot Imitation Learning by Querying Unlabeled Datasets](https://mlsanity.com/paper/2304.08742/) (2023)
- [QueST: Self-Supervised Skill Abstractions for Learning Continuous Control](https://mlsanity.com/paper/2407.15840/) (2024)
- [Imitation Learning by Estimating Expertise of Demonstrators](https://mlsanity.com/paper/2202.01288/) (2022)
- [Re-Mix: Optimizing Data Mixtures for Large Scale Imitation Learning](https://mlsanity.com/paper/2408.14037/) (2024)
- [Efficient Data Collection for Robotic Manipulation via Compositional Generalization](https://mlsanity.com/paper/2403.05110/) (2024)
- [PRISE: LLM-Style Sequence Compression for Learning Temporal Action Abstractions in Control](https://mlsanity.com/paper/2402.10450/) (2024)
- [So You Think You Can Scale Up Autonomous Robot Data Collection?](https://mlsanity.com/paper/2411.01813/) (2024)

## Built on by (most cited first, 18 of 18)

- [DataMIL: Selecting Data for Robot Imitation Learning with Datamodels](https://mlsanity.com/paper/2505.09603/) (2025)
- [CUPID: Curating Data your Robot Loves with Influence Functions](https://mlsanity.com/paper/2506.19121/) (2025)
- [World Action Verifier: Self-Improving World Models via Forward-Inverse Asymmetry](https://mlsanity.com/paper/2604.01985/) (2026)
- [SCIZOR: A Self-Supervised Approach to Data Curation for Large-Scale Imitation Learning](https://mlsanity.com/paper/2505.22626/) (2025)
- [X-Diffusion: Training Diffusion Policies on Cross-Embodiment Human Demonstrations](https://mlsanity.com/paper/2511.04671/) (2025)
- [Learning from the Best: Smoothness-Driven Metrics for Data Quality in Imitation Learning](https://mlsanity.com/paper/2604.23000/) (2026)
- [An Efficient Metric for Data Quality Measurement in Imitation Learning](https://mlsanity.com/paper/2605.01544/) (2026)
- [FACTR 2: Learning External Force Sensing for Commodity Robot Arms Improves Policy Learning](https://mlsanity.com/paper/2606.12406/) (2026)
- [Quality over Quantity: Demonstration Curation via Influence Functions for Data-Centric Robot Learning](https://mlsanity.com/paper/2603.09056/) (2026)
- [SPARC: Reliable Spatial Annotations from Robot Demonstrations at Scale](https://mlsanity.com/paper/2606.13497/) (2026)
- [Active Real-World Factor-Based Evaluation for Generalist Robot Policies](https://mlsanity.com/paper/2607.14439/) (2026)
- [Geometric Entropy: When Trajectory Diversity Helps and Hurts in Imitation Learning](https://mlsanity.com/paper/2606.20871/) (2026)
- [TSD: A Physics-Inspired Trajectory Saliency Detector for Efficient Imitation Learning](https://mlsanity.com/paper/2606.23371/) (2026)
- [Robo-ValueRL: Reliable Value Estimation for Offline-to-Online Reinforcement Learning](https://mlsanity.com/paper/2607.09866/) (2026)
- [Auditing Instruction-Trajectory Mismatches in Multimodal Robot Demonstrations](https://mlsanity.com/paper/2608.07895/) (2026)
- [No Free Checker: A Survey of Verifiers for Robot Policies](https://mlsanity.com/paper/2609.09250/) (2026)
- [NEEDLEWORK: Offline Rewriting of Robot Data with Verified Local Stitches](https://mlsanity.com/paper/2610.02339/) (2026)
- [Where Small Mistakes Matter: Quantifying State Criticality for Imitation Learning](https://mlsanity.com/paper/2610.17709/) (2026)

## Abstract (arXiv)

Many robot demonstration datasets contain heterogeneous demonstrations of varying quality. This heterogeneity may benefit policy pre-training, but can hinder robot performance when used with a final imitation learning objective. In particular, some strategies in the data may be less reliable than others or may be underrepresented in the data, leading to poor performance when such strategies are sampled at test time. Moreover, such unreliable or underrepresented strategies can be difficult even for people to discern, and sifting through demonstration datasets is time-consuming and costly. On the other hand, policy performance when trained on such demonstrations can reflect the reliability of different strategies. We thus propose for robots to self-curate based on online robot experience (Demo-SCORE). More specifically, we train and cross-validate a classifier to discern successful policy roll-outs from unsuccessful ones and use the classifier to filter heterogeneous demonstration datasets. Our experiments in simulation and the real world show that Demo-SCORE can effectively identify suboptimal demonstrations without manual curation. Notably, Demo-SCORE achieves over 15-35% higher absolute success rate in the resulting policy compared to the base policy trained with all original demonstrations.