# Dancing with Still Images: Video Distillation via Static-Dynamic Disentanglement

arXiv: https://arxiv.org/abs/2312.00362 · HTML: https://mlsanity.com/paper/2312.00362/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 30 references that are in the map.
- First posted on arXiv 1 Dec 2023.

## Builds on (most cited first, 19 of 19)

- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [Is Space-Time Attention All You Need for Video Understanding?](https://mlsanity.com/paper/2102.05095/) (2021)
- [VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Training](https://mlsanity.com/paper/2203.12602/) (2022)
- [On the Properties of Neural Machine Translation: Encoder-Decoder Approaches](https://mlsanity.com/paper/1409.1259/) (2014)
- [A Closer Look at Spatiotemporal Convolutions for Action Recognition](https://mlsanity.com/paper/1711.11248/) (2017)
- [Dataset Condensation with Gradient Matching](https://mlsanity.com/paper/2006.05929/) (2020)
- [Dataset Distillation by Matching Training Trajectories](https://mlsanity.com/paper/2203.11932/) (2022)
- [Dataset Condensation with Distribution Matching](https://mlsanity.com/paper/2110.04181/) (2021)
- [Dataset Condensation with Differentiable Siamese Augmentation](https://mlsanity.com/paper/2102.08259/) (2021)
- [Dataset Meta-Learning from Kernel Ridge-Regression](https://mlsanity.com/paper/2011.00050/) (2020)
- [Dataset Distillation with Infinitely Wide Convolutional Networks](https://mlsanity.com/paper/2107.13034/) (2021)
- [Dataset Condensation via Efficient Synthetic-Data Parameterization](https://mlsanity.com/paper/2205.14959/) (2022)
- [Scaling Up Dataset Distillation to ImageNet-1K with Constant Memory](https://mlsanity.com/paper/2211.10586/) (2022)
- [Dataset Distillation using Neural Feature Regression](https://mlsanity.com/paper/2206.00719/) (2022)
- [Dataset Distillation via Factorization](https://mlsanity.com/paper/2210.16774/) (2022)
- [Efficient Dataset Distillation Using Random Feature Approximation](https://mlsanity.com/paper/2210.12067/) (2022)
- [Remember the Past: Distilling Datasets into Addressable Memories for Neural Networks](https://mlsanity.com/paper/2206.02916/) (2022)
- [Data Distillation for Text Classification](https://mlsanity.com/paper/2104.08448/) (2021)
- [No frame left behind: Full Video Action Recognition](https://mlsanity.com/paper/2103.15395/) (2021)

## Built on by (most cited first, 21 of 21)

- [The Evolution of Dataset Distillation: Toward Scalable and Generalizable Solutions](https://mlsanity.com/paper/2502.05673/) (2025)
- [Beyond Modality Collapse: Representations Blending for Multimodal Dataset Distillation](https://mlsanity.com/paper/2505.14705/) (2025)
- [A Large-Scale Study on Video Action Dataset Condensation](https://mlsanity.com/paper/2412.21197/) (2024)
- [Distilling Dataset into Neural Field](https://mlsanity.com/paper/2503.04835/) (2025)
- [Condensing Action Segmentation Datasets via Generative Network Inversion](https://mlsanity.com/paper/2503.14112/) (2025)
- [CONCORD: Concept-Informed Diffusion for Dataset Distillation](https://mlsanity.com/paper/2505.18358/) (2025)
- [CovMatch: Cross-Covariance Guided Multimodal Dataset Distillation with Trainable Text Encoder](https://mlsanity.com/paper/2510.18583/) (2025)
- [VideoCompressa: Data-Efficient Video Understanding via Joint Temporal Compression and Spatial Reconstruction](https://mlsanity.com/paper/2511.18831/) (2025)
- [PRISM: Video Dataset Condensation with Progressive Refinement and Insertion for Sparse Motion](https://mlsanity.com/paper/2505.22564/) (2025)
- [Parameterizing Dataset Distillation via Gaussian Splatting](https://mlsanity.com/paper/2509.26219/) (2025)
- [Latent Video Dataset Distillation](https://mlsanity.com/paper/2504.17132/) (2025)
- [Temporal Saliency-Guided Distillation: A Scalable Framework for Distilling Video Datasets](https://mlsanity.com/paper/2505.20694/) (2025)
- [GVD: Guiding Video Diffusion Model for Scalable Video Distillation](https://mlsanity.com/paper/2507.22360/) (2025)
- [Relational Database Distillation: From Structured Tables to Condensed Graph Data](https://mlsanity.com/paper/2510.06980/) (2025)
- [State Diversity Matters in Offline Behavior Distillation](https://mlsanity.com/paper/2512.06692/) (2025)
- [Multimodal Dataset Distillation via Phased Teacher Models](https://mlsanity.com/paper/2603.25388/) (2026)
- [Video Dataset Condensation with Diffusion Models](https://mlsanity.com/paper/2505.06670/) (2025)
- [Dynamic-Aware Video Distillation: Optimizing Temporal Resolution Based on Video Semantics](https://mlsanity.com/paper/2506.02021/) (2025)
- [InfoMotion: A Graph-Based Approach to Video Dataset Distillation for Echocardiography](https://mlsanity.com/paper/2512.09422/) (2025)
- [Distill Video Datasets into Images](https://mlsanity.com/paper/2512.14621/) (2025)
- [Adaptive Latent Trajectory Anchoring for Action Segmentation Dataset Condensation](https://mlsanity.com/paper/2607.09081/) (2026)

## Abstract (arXiv)

Recently, dataset distillation has paved the way towards efficient machine learning, especially for image datasets. However, the distillation for videos, characterized by an exclusive temporal dimension, remains an underexplored domain. In this work, we provide the first systematic study of video distillation and introduce a taxonomy to categorize temporal compression. Our investigation reveals that the temporal information is usually not well learned during distillation, and the temporal dimension of synthetic data contributes little. The observations motivate our unified framework of disentangling the dynamic and static information in the videos. It first distills the videos into still images as static memory and then compensates the dynamic and motion information with a learnable dynamic memory block. Our method achieves state-of-the-art on video datasets at different scales, with a notably smaller memory storage budget. Our code is available at https://github.com/yuz1wan/video_distillation.