mlsanityData to 14 Oct 2026

Counterfactual Gate-Support Decomposition: Separating Selection from Redistribution in Training-Data Filtering

arXiv 2610.19343 ↗First posted 14 Oct 2026cs.LG cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

A training-data quality gate changes both which examples survive within regions of the input space and how training mass is distributed across those regions, so a gate-versus-random comparison does not isolate these mechanisms. We introduce Counterfactual Gate-Support Decomposition (CGSD), a four-arm, budget-matched design that crosses gate eligibility with support distribution and decomposes the observed gate effect into within-support selection, between-support redistribution, and their interaction. The practical constraint is finite-pool realizability: we characterize mass and token-assembly conditions under which the required arms can be constructed, together with a support-extinction case in which the ordinary design is undefined. On Korean-English and Korean-Japanese retrieval, the mean within-support selection component is the only one that exceeds its calibrated null band (+29.7 and +5.6 holdout items; 30/30 positive replicates), decisively in Korean-English and marginally in Korean-Japanese; redistribution and interaction remain unresolved at the available power. The same pattern holds for a second, unrelated gate family -- a model-confidence threshold rather than a deterministic verifier -- judged against its own null (+27.3 items, 9/9 positive), so the result is not a property of one gate's mechanism. On a mathematics gate, zero-eligibility supports leave the ordinary design undefined at every budget, while the extended design is constructible only inside narrow budget bands that an exhaustive scan locates and that the natural operating point misses -- the assembly condition separating a property of the pool from a limitation of the packer. CGSD therefore returns a decomposition, a constructible-but-underpowered design, or an explicit non-constructibility result, and says which; it prevents support redistribution from being silently interpreted as sample-level gate quality.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 24 of its 30 references that are in the map. First posted on arXiv 14 Oct 2026.

Builds on

24 of 30 references.

References · 24

  1. 2017Understanding Black-box Predictions via Influence Functionscited by 1,741
  2. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 807
  3. 2017Deep Reinforcement Learning that Matterscited by 754
  4. 2021Deduplicating Training Data Makes Language Models Bettercited by 714
  5. 2019Data Shapley: Equitable Valuation of Data for Machine Learningcited by 629
  6. 2020Estimating Training Data Influence by Tracing Gradient Descentcited by 580
  7. 2024LESS: Selecting Influential Data for Targeted Instruction Tuningcited by 532
  8. 2022Beyond neural scaling laws: beating power law scaling via data pruningcited by 528
  9. 2023DoReMi: Optimizing Data Mixtures Speeds Up Language Model Pretrainingcited by 399
  10. 2023Data Selection for Language Models via Importance Resamplingcited by 329
  11. 2023Studying Large Language Model Generalization with Influence Functionscited by 326
  12. 2020Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program)cited by 322
12 more
  1. 2023SemDeDup: Data-efficient learning at web-scale through semantic deduplicationcited by 303
  2. 2013Direct and Indirect Effectscited by 280
  3. 2021Accounting for Variance in Machine Learning Benchmarkscited by 201
  4. 2022Datamodels: Predicting Predictions from Training Datacited by 189
  5. 2024Data Mixing Laws: Optimizing Data Mixtures by Predicting Language Modeling Performancecited by 184
  6. 2024RegMix: Data Mixture as Regression for Language Model Pre-trainingcited by 177
  7. 2019Show Your Work: Improved Reporting of Experimental Resultscited by 169
  8. 2024DsDm: Model-Aware Dataset Selection with Datamodelscited by 105
  9. 2023DoGE: Domain Reweighting with Generalization Estimationcited by 90
  10. 2021Torch.manual_seed(3407) is all you need: On the influence of random seeds in deep learning architectures for computer visioncited by 81
  11. 2024Data Shapley in One Training Runcited by 81
  12. 2024Aioli: A Unified Optimization Framework for Language Model Data Mixingcited by 34

Built on by

Not cited yet.