Geometric Median (GM) Matching for Robust Data Pruning
Not on the map. Why?
Abstract (arXiv)
Large-scale data collections in the wild, are invariably noisy. Thus developing data pruning strategies that remain robust even in the presence of corruption is critical in practice. In this work, we propose Geometric Median (gm) Matching -- a herding style greedy algorithm that yields a k-subset such that the mean of the subset approximates the geometric median of the (potentially) noisy dataset. Theoretically, we show that gm Matching enjoys an improved gO(1/k) scaling over gO(1/√k) scaling of uniform sampling; while achieving bf optimal breakdown point of bf 1/2 even under bf arbitrary corruption. Extensive experiments across several popular deep learning benchmarks indicate that gm Matching consistently improves over prior state-of-the-art; the gains become more profound at high rates of corruption and aggressive pruning rates; making gm Matching a strong baseline for future research in robust data pruning.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 34 of its 66 references that are in the map. First posted on arXiv 25 Jun 2024.
Builds on
34 of 66 references.
References · 34
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
- 2014Explaining and Harnessing Adversarial Examplescited by 7,287
- 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
- 2020Scaling Laws for Neural Language Modelscited by 6,411
- 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
- 2013Intriguing properties of neural networkscited by 5,114
- 2017Squeeze-and-Excitation Networkscited by 4,763
- 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,857
- 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,388
22 more
- 2018ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Designcited by 1,132
- 2018Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labelscited by 994
- 2018An Empirical Study of Example Forgetting during Deep Neural Network Learningcited by 669
- 2016Making Deep Neural Networks Robust to Label Noise: a Loss Correction Approachcited by 663
- 2017MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labelscited by 652
- 2017Deep Learning Scaling is Predictable, Empiricallycited by 556
- 2021Deep Learning on a Data Diet: Finding Important Examples Early in Trainingcited by 529
- 2022Beyond neural scaling laws: beating power law scaling via data pruningcited by 519
- 2018Not All Samples Are Created Equal: Deep Learning with Importance Samplingcited by 323
- 2018Characterizing Adversarial Subspaces Using Local Intrinsic Dimensionalitycited by 280
- 2020Identifying Mislabeled Data using the Area Under the Margin Rankingcited by 259
- 2020A Theory of Usable Information Under Computational Constraintscited by 153
- 2018RSA: Byzantine-Robust Stochastic Aggregation Methods for Distributed Learning from Heterogeneous Datasetscited by 148
- 2022Dataset Pruning: Reducing Training Data by Examining Generalization Influencecited by 132
- 2022Selective-Supervised Contrastive Learning with Noisy Labelscited by 112
- 2019Recent Advances in Algorithmic High-Dimensional Robust Statisticscited by 100
- 2019Federated Variance-Reduced Stochastic Gradient Descent with Robustness to Byzantine Attackscited by 79
- 2016Geometric Median in Nearly Linear Timecited by 38
- 2020Choosing the Sample with Lowest Loss makes SGD Robustcited by 26
- 2023Data-Efficient Contrastive Self-supervised Learning: Most Beneficial Examples for Supervised Learning Contribute the Leastcited by 22
- 2021Robust Training in High Dimensions via Block Coordinate Geometric Median Descentcited by 20
- 2023Robust Data Pruning under Label Noise via Maximizing Re-labeling Accuracycited by 20
Built on by
Citing papers per year: 2025: 1.
1 paper.