# CLASS: Contrastive Learning via Action Sequence Supervision for Robot Manipulation

arXiv: https://arxiv.org/abs/2508.01600 · HTML: https://mlsanity.com/paper/2508.01600/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Aug 2025.

## Built on by (most cited first, 11 of 11)

- [Contrastive Representation Regularization for Vision-Language-Action Models](https://mlsanity.com/paper/2510.01711/) (2025)
- [Do You Know Where Your Camera Is? View-Invariant Policy Learning with Camera Conditioning](https://mlsanity.com/paper/2510.02268/) (2025)
- [Learning to Act Robustly with View-Invariant Latent Actions](https://mlsanity.com/paper/2601.02994/) (2026)
- [World2Act: Latent Action Post-Training from World Model Dynamics](https://mlsanity.com/paper/2603.10422/) (2026)
- [FAR: Failure-Aware Retry for Test-Time Recovery and Continual Policy Improvement](https://mlsanity.com/paper/2607.01111/) (2026)
- [CLAMP: Contrastive Learning for 3D Multi-View Action-Conditioned Robotic Manipulation Pretraining](https://mlsanity.com/paper/2602.00937/) (2026)
- [FOCA: Future-Oriented Conditioning for Data-Efficient Vision-Language-Action Adaptation](https://mlsanity.com/paper/2606.20867/) (2026)
- [Contrastive Action-Image Pre-training for Visuomotor Control](https://mlsanity.com/paper/2606.17256/) (2026)
- [RayViT: Ray-Conditioned Visual Representations for Viewpoint-Robust Imitation Learning](https://mlsanity.com/paper/2607.29622/) (2026)
- [An Empirical Study on What Matters for Viewpoint-Generalizable Policies in Visual Imitation Learning](https://mlsanity.com/paper/2609.32762/) (2026)
- [RARA: Robust Policy Fine-Tuning through Retrieval-Anchored Representation Alignment for Robotic Manipulation](https://mlsanity.com/paper/2610.13578/) (2026)

## Abstract (arXiv)

Recent advances in Behavior Cloning (BC) have led to strong performance in robotic manipulation, driven by expressive models, sequence modeling of actions, and large-scale demonstration data. However, BC faces significant challenges when applied to heterogeneous datasets, such as visual shift with different camera poses or object appearances, where performance degrades despite the benefits of learning at scale. This stems from BC's tendency to overfit individual demonstrations rather than capture shared structure, limiting generalization. To address this, we introduce Contrastive Learning via Action Sequence Supervision (CLASS), a method for learning behavioral representations from demonstrations using supervised contrastive learning. CLASS leverages weak supervision from similar action sequences identified via Dynamic Time Warping (DTW) and optimizes a soft InfoNCE loss with similarity-weighted positive pairs. We evaluate CLASS on 5 simulation benchmarks and 3 real-world tasks to achieve competitive results using retrieval-based control with representations only. Most notably, for downstream policy learning under significant visual shifts, Diffusion Policy with CLASS pre-training achieves an average success rate of 75%, while all other baseline methods fail to perform competitively. Project webpage: https://class-robot.github.io.