# Mind Your Margin and Boundary: Are Your Distilled Datasets Truly Robust?

arXiv: https://arxiv.org/abs/2605.20606 · HTML: https://mlsanity.com/paper/2605.20606/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 1 of its 47 references that are in the map.
- First posted on arXiv 20 May 2026.

## Builds on (most cited first, 1 of 1)

- [A Unified Framework for Knowledge Transfer in Bidirectional Model Scaling](https://mlsanity.com/paper/2603.07506/) (2026)

## Built on by (most cited first, 1 of 1)

- [All for 1-Bit: Towards Genuine 1-Bit Post-Training Quantization for LLMs](https://mlsanity.com/paper/2609.06161/) (2026)

## Abstract (arXiv)

Dataset distillation (DD) compresses a large training set into a small synthetic set for efficient training, but most DD methods optimize only clean accuracy and leave robustness uncontrolled. Recent robust DD methods improve robustness, yet they often suffer from a poor accuracy-robustness trade-off because they (i) treat all adversarially perturbed examples uniformly, despite robust risk being dominated by near-zero robust margins, and (ii) do not explicitly increase inter-class separation in the decision boundary where attacks concentrate. We present Contrastive Curriculum for Robust Dataset Distillation (C2R), a framework that couples an attack-aware curriculum with a contrastive robustness objective. From a robust-margin perspective, we derive a perturbation score that approximates each sample's robust hinge, enabling a curriculum that prioritizes the smallest-margin adversaries that most directly drive robust error. In parallel, a class-balanced contrastive robustness loss enforces adversarial invariance while explicitly widening boundary separation across classes. Experiments on CIFAR-10/100, Tiny-ImageNet, and multiple ImageNet-1K subsets under six attacks show that C2R achieves the best robust accuracy, outperforming prior robust DD by 2.8% on average.