# Progressive trajectory matching for medical dataset distillation

arXiv: https://arxiv.org/abs/2403.13469 · HTML: https://mlsanity.com/paper/2403.13469/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 34 references that are in the map.
- First posted on arXiv 20 Mar 2024.

## Builds on (most cited first, 24 of 24)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [Visualizing the Loss Landscape of Neural Nets](https://mlsanity.com/paper/1712.09913/) (2017)
- [MedMNIST v2 -- A large-scale lightweight benchmark for 2D and 3D biomedical image classification](https://mlsanity.com/paper/2110.14795/) (2021)
- [Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNs](https://mlsanity.com/paper/1802.10026/) (2018)
- [Dataset Condensation with Gradient Matching](https://mlsanity.com/paper/2006.05929/) (2020)
- [The Liver Tumor Segmentation Benchmark (LiTS)](https://mlsanity.com/paper/1901.04056/) (2019)
- [Dataset Distillation by Matching Training Trajectories](https://mlsanity.com/paper/2203.11932/) (2022)
- [Dataset Condensation with Distribution Matching](https://mlsanity.com/paper/2110.04181/) (2021)
- [Dataset Condensation with Differentiable Siamese Augmentation](https://mlsanity.com/paper/2102.08259/) (2021)
- [Minimizing the Accumulated Trajectory Error to Improve Dataset Distillation](https://mlsanity.com/paper/2211.11004/) (2022)
- [Dataset Distillation: A Comprehensive Review](https://mlsanity.com/paper/2301.07014/) (2023)
- [Towards Lossless Dataset Distillation via Difficulty-Aligned Trajectory Matching](https://mlsanity.com/paper/2310.05773/) (2023)
- [Dataset Distillation via Factorization](https://mlsanity.com/paper/2210.16774/) (2022)
- [A Comprehensive Survey of Dataset Distillation](https://mlsanity.com/paper/2301.05603/) (2023)
- [Data Distillation: A Survey](https://mlsanity.com/paper/2301.04272/) (2023)
- [The History Began from AlexNet: A Comprehensive Survey on Deep Learning Approaches](https://mlsanity.com/paper/1803.01164/) (2018)
- [DC-BENCH: Dataset Condensation Benchmark](https://mlsanity.com/paper/2207.09639/) (2022)
- [Soft-Label Anonymous Gastric X-ray Image Distillation](https://mlsanity.com/paper/2104.02857/) (2021)
- [Sequential Subset Matching for Dataset Distillation](https://mlsanity.com/paper/2311.01570/) (2023)
- [Compressed Gastric Image Generation Based on Soft-Label Dataset Distillation for Medical Data Sharing](https://mlsanity.com/paper/2209.14635/) (2022)
- [A Survey on Dataset Distillation: Approaches, Applications and Future Directions](https://mlsanity.com/paper/2305.01975/) (2023)
- [Dataset Distillation for Medical Dataset Sharing](https://mlsanity.com/paper/2209.14603/) (2022)
- [ConvNets Match Vision Transformers at Scale](https://mlsanity.com/paper/2310.16764/) (2023)

## Built on by (most cited first, 5 of 5)

- [The Evolution of Dataset Distillation: Toward Scalable and Generalizable Solutions](https://mlsanity.com/paper/2502.05673/) (2025)
- [A Survey on Trustworthiness in Foundation Models for Medical Image Analysis](https://mlsanity.com/paper/2407.15851/) (2024)
- [Evolution-aware VAriance (EVA) Coreset Selection for Medical Image Classification](https://mlsanity.com/paper/2406.05677/) (2024)
- [Label-Consistent Dataset Distillation with Detector-Guided Refinement](https://mlsanity.com/paper/2507.13074/) (2025)
- [Federated Distillation for Whole Slide Image via Gaussian-Mixture Feature Alignment and Curriculum Integration](https://mlsanity.com/paper/2605.00578/) (2026)

## Abstract (arXiv)

It is essential but challenging to share medical image datasets due to privacy issues, which prohibit building foundation models and knowledge transfer. In this paper, we propose a novel dataset distillation method to condense the original medical image datasets into a synthetic one that preserves useful information for building an analysis model without accessing the original datasets. Existing methods tackle only natural images by randomly matching parts of the training trajectories of the model parameters trained by the whole real datasets. However, through extensive experiments on medical image datasets, the training process is extremely unstable and achieves inferior distillation results. To solve these barriers, we propose to design a novel progressive trajectory matching strategy to improve the training stability for medical image dataset distillation. Additionally, it is observed that improved stability prevents the synthetic dataset diversity and final performance improvements. Therefore, we propose a dynamic overlap mitigation module that improves the synthetic dataset diversity by dynamically eliminating the overlap across different images and retraining parts of the synthetic images for better convergence. Finally, we propose a new medical image dataset distillation benchmark of various modalities and configurations to promote fair evaluations. It is validated that our proposed method achieves 8.33% improvement over previous state-of-the-art methods on average, and 11.7% improvement when ipc=2 (i.e., image per class is 2). Codes and benchmarks will be released.