Useful to Whom? Sample Value Is Defined Only Relative to the Learner
Not on the map. Why?
Abstract (arXiv)
What kind of data does a model need in order to learn? Coreset selection makes this question concrete: under a budget, keep the samples most useful for training. Easy-first and geometric coverage criteria can win in different budget regimes, separated by a crossover boundary. We ask whether this boundary is fixed by the data or changes with the target learner. Controlled experiments freeze the selected subsets and manipulate only the training learner. On low-resolution ImageNet-100, doubling ResNet-18's width moves the crossover from 57 to 85 samples per class: the learner changes the relative value of the same samples. A wider sweep reveals an interaction between input grid and capacity. Enlarging the grid while retaining the same image information shifts the boundary left, and this shift weakens as width increases. Stride controls reproduce and reverse the grid effect without changing the input grid; removing only the last downsampling stride is sufficient to recover the leftward shift. Under the native-224px ImageNet-1k protocol, width effects are smaller and depend on the probe: LFrac remains nearly flat, while EL2N shifts modestly right. Swapping the convolutional learning system for a ViT makes coverage win throughout the measured range, even when the easy subsets come from the convolutional proxy. These results establish learner dependence through frozen-subset interventions and identify network structure that can move the boundary. They do not yield a universal scaling law. Their practical implication is direct: a selection strategy's preferred budget regime must be evaluated with respect to the target learner.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 15 of its 21 references that are in the map. First posted on arXiv 22 Sep 2026.
Builds on
15 of 21 references.
References · 15
- 2017Understanding Black-box Predictions via Influence Functionscited by 1,714
- 2018An Empirical Study of Example Forgetting during Deep Neural Network Learningcited by 669
- 2019Data Shapley: Equitable Valuation of Data for Machine Learningcited by 617
- 2021Deep Learning on a Data Diet: Finding Important Examples Early in Trainingcited by 529
- 2022Beyond neural scaling laws: beating power law scaling via data pruningcited by 519
- 2020Dataset Cartography: Mapping and Diagnosing Datasets with Training Dynamicscited by 423
- 2019Selection via Proxy: Efficient Data Selection for Deep Learningcited by 337
- 2021GRAD-MATCH: Gradient Matching based Data Subset Selection for Efficient Deep Model Trainingcited by 269
- 2022DeepCore: A Comprehensive Library for Coreset Selection in Deep Learningcited by 146
- 2021Deep Learning Through the Lens of Example Difficultycited by 139
- 2022Coverage-centric Coreset Selection for High Pruning Ratescited by 115
- 2020Characterizing Structural Regularities of Labeled Data in Overparameterized Modelscited by 93
Built on by
Not cited yet.