# Dataset Distillation with Neural Characteristic Function: A Minmax Perspective

arXiv: https://arxiv.org/abs/2502.20653 · HTML: https://mlsanity.com/paper/2502.20653/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 61 references that are in the map.
- First posted on arXiv 28 Feb 2025.

## Builds on (most cited first, 30 of 40)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks](https://mlsanity.com/paper/1511.06434/) (2015)
- [Demystifying MMD GANs](https://mlsanity.com/paper/1801.01401/) (2018)
- [An Empirical Study of Example Forgetting during Deep Neural Network Learning](https://mlsanity.com/paper/1812.05159/) (2018)
- [Dataset Condensation with Gradient Matching](https://mlsanity.com/paper/2006.05929/) (2020)
- [Towards Principled Methods for Training Generative Adversarial Networks](https://mlsanity.com/paper/1701.04862/) (2017)
- [Dataset Distillation by Matching Training Trajectories](https://mlsanity.com/paper/2203.11932/) (2022)
- [Dataset Condensation with Distribution Matching](https://mlsanity.com/paper/2110.04181/) (2021)
- [Dataset Condensation with Differentiable Siamese Augmentation](https://mlsanity.com/paper/2102.08259/) (2021)
- [MMD GAN: Towards Deeper Understanding of Moment Matching Network](https://mlsanity.com/paper/1705.08584/) (2017)
- [Dataset Meta-Learning from Kernel Ridge-Regression](https://mlsanity.com/paper/2011.00050/) (2020)
- [Dataset Condensation via Efficient Synthetic-Data Parameterization](https://mlsanity.com/paper/2205.14959/) (2022)
- [fastai: A Layered API for Deep Learning](https://mlsanity.com/paper/2002.04688/) (2020)
- [Scaling Up Dataset Distillation to ImageNet-1K with Constant Memory](https://mlsanity.com/paper/2211.10586/) (2022)
- [Dataset Distillation using Neural Feature Regression](https://mlsanity.com/paper/2206.00719/) (2022)
- [Minimizing the Accumulated Trajectory Error to Improve Dataset Distillation](https://mlsanity.com/paper/2211.11004/) (2022)
- [Squeeze, Recover and Relabel: Dataset Condensation at ImageNet Scale From A New Perspective](https://mlsanity.com/paper/2306.13092/) (2023)
- [Improved Distribution Matching for Dataset Condensation](https://mlsanity.com/paper/2307.09742/) (2023)
- [Privacy for Free: How does Dataset Condensation Help Privacy?](https://mlsanity.com/paper/2206.00240/) (2022)
- [Towards Lossless Dataset Distillation via Difficulty-Aligned Trajectory Matching](https://mlsanity.com/paper/2310.05773/) (2023)
- [On the Diversity and Realism of Distilled Dataset: An Efficient Dataset Distillation Paradigm](https://mlsanity.com/paper/2312.03526/) (2023)
- [Dataset Condensation with Contrastive Signals](https://mlsanity.com/paper/2202.02916/) (2022)
- [Generative Teaching Networks: Accelerating Neural Architecture Search by Learning to Generate Synthetic Training Data](https://mlsanity.com/paper/1912.07768/) (2019)
- [DataDAM: Efficient Dataset Distillation with Attention Matching](https://mlsanity.com/paper/2310.00093/) (2023)
- [Generalized Large-Scale Data Condensation via Various Backbone and Statistical Matching](https://mlsanity.com/paper/2311.17950/) (2023)
- [M3D: Dataset Condensation by Minimizing Maximum Mean Discrepancy](https://mlsanity.com/paper/2312.15927/) (2023)
- [Private Set Generation with Discriminative Information](https://mlsanity.com/paper/2211.04446/) (2022)
- [Exploiting Inter-sample and Inter-feature Relations in Dataset Distillation](https://mlsanity.com/paper/2404.00563/) (2024)

## Built on by (most cited first, 30 of 45)

- [Shifting AI Efficiency From Model-Centric to Data-Centric Compression](https://mlsanity.com/paper/2505.19147/) (2025)
- [Data Whisperer: Efficient Data Selection for Task-Specific LLM Fine-Tuning via Few-Shot In-Context Learning](https://mlsanity.com/paper/2505.12212/) (2025)
- [CaO2: Rectifying Inconsistencies in Diffusion-Based Dataset Distillation](https://mlsanity.com/paper/2506.22637/) (2025)
- [Continual Learning for Generative AI: From LLMs to MLLMs and Beyond](https://mlsanity.com/paper/2506.13045/) (2025)
- [DD-Ranking: Rethinking the Evaluation of Dataset Distillation](https://mlsanity.com/paper/2505.13300/) (2025)
- [Beyond Modality Collapse: Representations Blending for Multimodal Dataset Distillation](https://mlsanity.com/paper/2505.14705/) (2025)
- [Efficient Multimodal Dataset Distillation via Generative Models](https://mlsanity.com/paper/2509.15472/) (2025)
- [Beyond Random: Automatic Inner-loop Optimization in Dataset Distillation](https://mlsanity.com/paper/2510.04838/) (2025)
- [Optimizing Distributional Geometry Alignment with Optimal Transport for Generative Dataset Distillation](https://mlsanity.com/paper/2512.00308/) (2025)
- [A Survey on the Green Development of Large Models: From Resource-Efficient Architectures to Hardware-Software Co-Design](https://mlsanity.com/paper/2607.09084/) (2026)
- [ImagebindDC: Compressing Multi-modal Data with Imagebind-based Condensation](https://mlsanity.com/paper/2511.08263/) (2025)
- [VideoCompressa: Data-Efficient Video Understanding via Joint Temporal Compression and Spatial Reconstruction](https://mlsanity.com/paper/2511.18831/) (2025)
- [GeoDM: Geometry-aware Distribution Matching for Dataset Distillation](https://mlsanity.com/paper/2512.08317/) (2025)
- [DMGD: Train-Free Dataset Distillation with Semantic-Distribution Matching in Diffusion Models](https://mlsanity.com/paper/2605.03877/) (2026)
- [Reasoning Like an Economist: Post-Training on Economic Problems Induces Strategic Generalization in LLMs](https://mlsanity.com/paper/2506.00577/) (2025)
- [Parameterizing Dataset Distillation via Gaussian Splatting](https://mlsanity.com/paper/2509.26219/) (2025)
- [Learning from Dense Events: Towards Fast Spiking Neural Networks Training via Event Dataset Distillation](https://mlsanity.com/paper/2511.12095/) (2025)
- [FAST: Topology-Aware Frequency-Domain Distribution Matching for Coreset Selection](https://mlsanity.com/paper/2511.19476/) (2025)
- [Task-Oriented Data Synthesis and Control-Rectify Sampling for Remote Sensing Semantic Segmentation](https://mlsanity.com/paper/2512.16740/) (2025)
- [Instance Data Condensation for Image Super-Resolution](https://mlsanity.com/paper/2505.21099/) (2025)
- [Multimodal-Guided Dynamic Dataset Pruning for Robust and Efficient Data-Centric Learning](https://mlsanity.com/paper/2507.12750/) (2025)
- [On the Fast Adaptation of Delayed Clients in Decentralized Federated Learning: A Centroid-Aligned Distillation Approach](https://mlsanity.com/paper/2508.02993/) (2025)
- [A Discrepancy-Based Perspective on Dataset Condensation](https://mlsanity.com/paper/2509.10367/) (2025)
- [FT-NCFM: An Influence-Aware Data Distillation Framework for Efficient VLA Models](https://mlsanity.com/paper/2511.16233/) (2025)
- [Grounding and Enhancing Informativeness and Utility in Dataset Distillation](https://mlsanity.com/paper/2601.21296/) (2026)
- [Learning Adaptive Distribution Alignment with Neural Characteristic Function for Graph Domain Adaptation](https://mlsanity.com/paper/2602.10489/) (2026)
- [Multimodal Dataset Distillation via Phased Teacher Models](https://mlsanity.com/paper/2603.25388/) (2026)
- [DIVER: Diving Deeper into Distilled Data via Expressive Semantic Recovery](https://mlsanity.com/paper/2605.12649/) (2026)
- [Geometry-Aware Dataset Condensation for Diffusion Model Training](https://mlsanity.com/paper/2606.05883/) (2026)
- [Distill Once, Adapt Life-Long: Exploring Dataset Distillation for Continual Test-Time Adaptation](https://mlsanity.com/paper/2606.20196/) (2026)

## Abstract (arXiv)

Dataset distillation has emerged as a powerful approach for reducing data requirements in deep learning. Among various methods, distribution matching-based approaches stand out for their balance of computational efficiency and strong performance. However, existing distance metrics used in distribution matching often fail to accurately capture distributional differences, leading to unreliable measures of discrepancy. In this paper, we reformulate dataset distillation as a minmax optimization problem and introduce Neural Characteristic Function Discrepancy (NCFD), a comprehensive and theoretically grounded metric for measuring distributional differences. NCFD leverages the Characteristic Function (CF) to encapsulate full distributional information, employing a neural network to optimize the sampling strategy for the CF's frequency arguments, thereby maximizing the discrepancy to enhance distance estimation. Simultaneously, we minimize the difference between real and synthetic data under this optimized NCFD measure. Our approach, termed Neural Characteristic Function Matching (mymethod), inherently aligns the phase and amplitude of neural features in the complex plane for both real and synthetic data, achieving a balance between realism and diversity in synthetic samples. Experiments demonstrate that our method achieves significant performance gains over state-of-the-art methods on both low- and high-resolution datasets. Notably, we achieve a 20.5% accuracy boost on ImageSquawk. Our method also reduces GPU memory usage by over 300× and achieves 20× faster processing speeds compared to state-of-the-art methods. To the best of our knowledge, this is the first work to achieve lossless compression of CIFAR-100 on a single NVIDIA 2080 Ti GPU using only 2.3 GB of memory.