# Adaptive Latent Trajectory Anchoring for Action Segmentation Dataset Condensation

arXiv: https://arxiv.org/abs/2607.09081 · HTML: https://mlsanity.com/paper/2607.09081/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 23 of its 33 references that are in the map.
- First posted on arXiv 10 Jul 2026.

## Builds on (most cited first, 23 of 23)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [Dataset Distillation by Matching Training Trajectories](https://mlsanity.com/paper/2203.11932/) (2022)
- [MS-TCN: Multi-Stage Temporal Convolutional Network for Action Segmentation](https://mlsanity.com/paper/1903.01945/) (2019)
- [Dataset Condensation with Distribution Matching](https://mlsanity.com/paper/2110.04181/) (2021)
- [Dataset Distillation](https://mlsanity.com/paper/1811.10959/) (2018)
- [Scaling Up Dataset Distillation to ImageNet-1K with Constant Memory](https://mlsanity.com/paper/2211.10586/) (2022)
- [ASFormer: Transformer for Action Segmentation](https://mlsanity.com/paper/2110.08568/) (2021)
- [Dataset Distillation via Factorization](https://mlsanity.com/paper/2210.16774/) (2022)
- [Temporal Action Segmentation: An Analysis of Modern Techniques](https://mlsanity.com/paper/2210.10352/) (2022)
- [Diffusion Action Segmentation](https://mlsanity.com/paper/2303.17959/) (2023)
- [Unsupervised Action Segmentation by Joint Representation Learning and Online Clustering](https://mlsanity.com/paper/2105.13353/) (2021)
- [C2F-TCN: A Framework for Semi and Fully Supervised Temporal Action Segmentation](https://mlsanity.com/paper/2212.11078/) (2022)
- [Dancing with Still Images: Video Distillation via Static-Dynamic Disentanglement](https://mlsanity.com/paper/2312.00362/) (2023)
- [Iterative Contrast-Classify For Semi-supervised Temporal Action Segmentation](https://mlsanity.com/paper/2112.01402/) (2021)
- [Leveraging Action Affinity and Continuity for Semi-supervised Temporal Action Segmentation](https://mlsanity.com/paper/2207.08653/) (2022)
- [Temporal Action Segmentation with High-level Complex Activity Labels](https://mlsanity.com/paper/2108.06706/) (2021)
- [OnlineTAS: An Online Baseline for Temporal Action Segmentation](https://mlsanity.com/paper/2411.01122/) (2024)
- [CLOT: Closed Loop Optimal Transport for Unsupervised Action Segmentation](https://mlsanity.com/paper/2507.03539/) (2025)
- [Coherent Temporal Synthesis for Incremental Action Segmentation](https://mlsanity.com/paper/2403.06102/) (2024)
- [Condensing Action Segmentation Datasets via Generative Network Inversion](https://mlsanity.com/paper/2503.14112/) (2025)

## Abstract (arXiv)

Dataset condensation for action segmentation synthesizes compact, informative representations of long, untrimmed video datasets. The existing approach relies on Variational Autoencoders and an iterative latent optimization; it is computationally expensive and suffers from over-smoothed reconstructions and rigid temporal constraints. This paper proposes to shift the condensation paradigm from optimization-based inversion to deterministic latent mapping. By leveraging Denoising Diffusion Implicit Models, we represent action segments as continuous trajectories anchored by sparse latent points in the noise manifold. To maximize representational efficiency, we introduce an adaptive allocation mechanism that dynamically redistributes the anchoring budget based on segment-wise reconstruction difficulty. Extensive experiments demonstrate that our framework significantly outperforms state-of-the-art methods in segmentation performance across common datasets. Notably, our approach achieves performance parity with real data training while maintaining a condensation ratio of 2.4% on Breakfast dataset.