mlsanityData to 14 Oct 2026

Prevalence Determines Precision:Silent Contamination in Detector-Defined Datasets

arXiv 2609.11449 ↗First posted 10 Sep 2026cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Many ML datasets are constructed by running a detector, heuristic, or model over candidate pools; accepted items become labels. Dataset precision is then governed by true-positive prevalence in each pool via Bayes, not solely by detector quality. Using one instrument and period, we hold a detector-defined event dataset plus an independent official index labeling every detected item as real or phantom. One detector, three pools yield phantom rates 81.7%, 9.0%, and 0.0%. Transferring precision from the two high-rate pools to the low-rate pool predicts 0.955 versus measured 0.183, a +422% error; the Bayes expression predicts all three within 3.3%. The detected response curve is an exact convex combination of a true-event and a phantom component (residual 1.1e-16), with phantoms outnumbering true events 473 to 308, so contamination is a second signal with detector-inherited shape, not additive noise. Contamination direction depends on the estimator: on identical windows one statistic is diluted and another inflated because its denominator is also contaminated. A common normalization turns the estimator into a mean of ratios whose expectation need not exist; on the same 335 events it returns 0.40 where the well-defined estimator returns 0.10.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 4 of its 17 references that are in the map. First posted on arXiv 10 Sep 2026.

Builds on

4 of 17 references.

References · 4

  1. 2018Datasheets for Datasetscited by 1,386
  2. 2019Pseudo-Labeling and Confirmation Bias in Deep Semi-Supervised Learningcited by 510
  3. 2021Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarkscited by 395
  4. 2017Snorkel: Rapid Training Data Creation with Weak Supervisioncited by 372

Built on by

Not cited yet.