mlsanityData to 7 Oct 2026

BWS: Best Window Selection Based on Sample Scores for Data Pruning across Broad Ranges

arXiv 2406.03057 ↗First posted 5 Jun 2024cs.LG stat.MLcited by 8

Not on the map. Why?

Abstract (arXiv)

Data subset selection aims to find a smaller yet informative subset of a large dataset that can approximate the full-dataset training, addressing challenges associated with training neural networks on large-scale datasets. However, existing methods tend to specialize in either high or low selection ratio regimes, lacking a universal approach that consistently achieves competitive performance across a broad range of selection ratios. We introduce a universal and efficient data subset selection method, Best Window Selection (BWS), by proposing a method to choose the best window subset from samples ordered based on their difficulty scores. This approach offers flexibility by allowing the choice of window intervals that span from easy to difficult samples. Furthermore, we provide an efficient mechanism for selecting the best window subset by evaluating its quality using kernel ridge regression. Our experimental results demonstrate the superior performance of BWS compared to other baselines across a broad range of selection ratios over datasets, including CIFAR-10/100 and ImageNet, and the scenarios involving training from random initialization or fine-tuning of pre-trained models.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 34 of its 49 references that are in the map. First posted on arXiv 5 Jun 2024.

Builds on

34 of 49 references.

References · 34

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  4. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,597
  5. 2017Understanding Black-box Predictions via Influence Functionscited by 1,714
  6. 2019Energy and Policy Considerations for Deep Learning in NLPcited by 1,242
  7. 2018An Empirical Study of Example Forgetting during Deep Neural Network Learningcited by 669
  8. 2019Green AIcited by 620
  9. 2019Data Shapley: Equitable Valuation of Data for Machine Learningcited by 617
  10. 2019On Exact Computation with an Infinitely Wide Neural Netcited by 572
  11. 2020Estimating Training Data Influence by Tracing Gradient Descentcited by 565
  12. 2021Deep Learning on a Data Diet: Finding Important Examples Early in Trainingcited by 529
22 more
  1. 2022Beyond neural scaling laws: beating power law scaling via data pruningcited by 519
  2. 2020What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimationcited by 492
  3. 2020Dataset Cartography: Mapping and Diagnosing Datasets with Training Dynamicscited by 423
  4. 2019Selection via Proxy: Efficient Data Selection for Deep Learningcited by 337
  5. 2020Identifying Mislabeled Data using the Area Under the Margin Rankingcited by 259
  6. 2020GLISTER: Generalization based Data Subset Selection for Efficient and Robust Learningcited by 224
  7. 2022Dataset Distillation using Neural Feature Regressioncited by 177
  8. 2021Beta Shapley: a Unified and Noise-reduced Data Valuation Framework for Machine Learningcited by 162
  9. 2022Coverage-centric Coreset Selection for High Pruning Ratescited by 115
  10. 2022Neural Collapse: A Review on Modelling Principles and Generalizationcited by 95
  11. 2020Spectra of the Conjugate Kernel and Neural Tangent Kernel for linear-width neural networkscited by 76
  12. 2023LAVA: Data Valuation without Pre-Specified Learning Algorithmscited by 69
  13. 2022Adaptive Second Order Coresets for Data-efficient Machine Learningcited by 60
  14. 2023Towards Sustainable Learning: Coresets for Data-efficient Deep Learningcited by 44
  15. 2020Efficient computation and analysis of distributional Shapley valuescited by 38
  16. 2022Characterizing Datapoints via Second-Split Forgettingcited by 37
  17. 2023Data Valuation Without Training of a Modelcited by 35
  18. 2023Loss-Curvature Matching for Dataset Selection and Condensationcited by 32
  19. 2024SelMatch: Effectively Scaling Up Dataset Distillation via Selection-Based Initialization and Partial Updates by Trajectory Matchingcited by 22
  20. 2021A Neural Pre-Conditioning Active Learning Algorithm to Reduce Label Complexitycited by 7
  21. 2021Finding High-Value Training Data Subset through Differentiable Convex Programmingcited by 5
  22. 2023Leveraging Importance Weights in Subset Selectioncited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 6; 2026: 1.

8 papers.

  1. 2025Lightweight Dataset Pruning without Full Training via Example Difficulty and Prediction Uncertaintycited by 22
  2. 2024ELFS: Label-Free Coreset Selection with Proxy Training Dynamicscited by 9
  3. 2025Coreset Selection via LLM-based Concept Bottleneckscited by 2
  4. 2025Class-Proportional Coreset Selection for Difficulty-Separable Datacited by 2
  5. 2025One Period to Rule Them All: Identifying Critical Learning Periods in Deep Networkscited by 1
  6. 2026Distill on a Diet: Efficient Knowledge Distillation via Learnable Data Pruningcited by 1
  7. 2025SNAP: Low-Latency Test-Time Adaptation with Sparse Updatescited by 0
  8. 2025Diffusion Reconstruction-based Data Likelihood Estimation for Core-Set Selectioncited by 0