mlsanityData to 14 Oct 2026

Detecting Errors in a Numerical Response via any Regression Model

arXiv 2305.16583 ↗First posted 26 May 2023stat.ML cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Noise plagues many numerical datasets, where the recorded values in the data may fail to match the true underlying values due to reasons including: erroneous sensors, data entry/processing mistakes, or imperfect human estimates. We consider general regression settings with covariates and a potentially corrupted response whose observed values may contain errors. By accounting for various uncertainties, we introduced veracity scores that distinguish between genuine errors and natural data fluctuations, conditioned on the available covariate information in the dataset. We propose a simple yet efficient filtering procedure for eliminating potential errors, and establish theoretical guarantees for our method. We also contribute a new error detection benchmark involving 5 regression datasets with real-world numerical errors (for which the true values are also known). In this benchmark and additional simulation studies, our method identifies incorrect values with better precision/recall than other approaches.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 19 of its 41 references that are in the map. First posted on arXiv 26 May 2023.

Builds on

19 of 41 references.

References · 19

  1. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  2. 2018Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labelscited by 997
  3. 2016Distribution-Free Predictive Inference For Regressioncited by 668
  4. 2017MentorNet: Learning Data-Driven Curriculum for Very Deep Neural Networks on Corrupted Labelscited by 653
  5. 2019Data Shapley: Equitable Valuation of Data for Machine Learningcited by 629
  6. 2020Learning from Noisy Labels with Deep Neural Networks: A Surveycited by 582
  7. 2019Confident Learning: Estimating Uncertainty in Dataset Labelscited by 413
  8. 2020AutoGluon-Tabular: Robust and Accurate AutoML for Structured Datacited by 383
  9. 2021Testing for Outliers with Conformal p-valuescited by 134
  10. 2019Distributional conformal predictioncited by 105
  11. 2020Estimating Example Difficulty Using Variance of Gradientscited by 97
  12. 2020CD-split and HPD-split: efficient conformal regions in high dimensionscited by 76
7 more
  1. 2017Converting High-Dimensional Regression to High-Dimensional Conditional Density Estimationcited by 34
  2. 2019Identifying Mislabeled Instances in Classification Datasetscited by 26
  3. 2022Back to the Basics: Revisiting Out-of-Distribution Detection Baselinescited by 16
  4. 2022Detecting Label Errors in Token Classification Datacited by 12
  5. 2022Identifying Incorrect Annotations in Multi-Label Classification Datacited by 12
  6. 2018RFCDE: Random Forests for Conditional Density Estimationcited by 8
  7. 2022Finding Label and Model Errors in Perception Data With Learned Observation Assertionscited by 6

Built on by

Not cited yet.