mlsanityData to 7 Oct 2026

Error Discovery by Clustering Influence Embeddings

arXiv 2312.04712 ↗First posted 7 Dec 2023cs.LGcited by 7

Not on the map. Why?

Abstract (arXiv)

We present a method for identifying groups of test examples -- slices -- on which a model under-performs, a task now known as slice discovery. We formalize coherence -- a requirement that erroneous predictions, within a slice, should be wrong for the same reason -- as a key property that any slice discovery method should satisfy. We then use influence functions to derive a new slice discovery method, InfEmbed, which satisfies coherence by returning slices whose examples are influenced similarly by the training data. InfEmbed is simple, and consists of applying K-Means clustering to a novel representation we deem influence embeddings. We show InfEmbed outperforms current state-of-the-art methods on 2 benchmarks, and is effective for model debugging across several case studies.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 35 of its 51 references that are in the map. First posted on arXiv 7 Dec 2023.

Builds on

35 of 51 references.

References · 35

  1. 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
  2. 2017Understanding Black-box Predictions via Influence Functionscited by 1,714
  3. 2020WILDS: A Benchmark of in-the-Wild Distribution Shiftscited by 1,366
  4. 2020Energy-based Out-of-distribution Detectioncited by 1,296
  5. 2020Estimating Training Data Influence by Tracing Gradient Descentcited by 565
  6. 2021Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarkscited by 388
  7. 2017Preventing Fairness Gerrymandering: Auditing and Learning for Subgroup Fairnesscited by 381
  8. 2019Unmasking Clever Hans Predictors and Assessing What Machines Really Learncited by 359
  9. 2018Manipulating and Measuring Model Interpretabilitycited by 256
  10. 2018Multiaccuracy: Black-Box Post-Processing for Fairness in Classificationcited by 221
  11. 2020No Subclass Left Behind: Fine-Grained Robustness in Coarse-Grained Classification Problemscited by 208
  12. 2021Simple data balancing achieves competitive worst-group-accuracycited by 177
23 more
  1. 2022Domino: Discovering Systematic Errors with Cross-Modal Embeddingscited by 155
  2. 2019Interpretations are useful: penalizing explanations to align neural networks with prior knowledgecited by 144
  3. 2021Scaling Up Influence Functionscited by 130
  4. 2020COVID-19 Image Data Collection: Prospective Predictions Are the Futurecited by 100
  5. 2021Do Feature Attribution Methods Correctly Attribute Features?cited by 98
  6. 2021An Interpretability Illusion for BERTcited by 94
  7. 2020Evaluating Saliency Map Explanations for Convolutional Neural Networks: A User Studycited by 86
  8. 2021Influence Selection for Active Learningcited by 82
  9. 2022Distilling Model Failures as Directions in Latent Spacecited by 82
  10. 2021The Spotlight: A General Method for Discovering Systematic Errors in Deep Learning Modelscited by 63
  11. 2022Post hoc Explanations may be Ineffective for Detecting Unknown Spurious Correlationcited by 59
  12. 2020Understanding Failures of Deep Networks via Robust Feature Extractioncited by 56
  13. 2018Deep Learning Predicts Hip Fracture using Confounding Patient and Healthcare Variablescited by 50
  14. 2021Interactive Label Cleaning with Example-based Explanationscited by 48
  15. 2022Discovering Bugs in Vision Models using Off-the-shelf Image Generation and Captioningcited by 45
  16. 2022The Road to Explainability is Paved with Bias: Measuring the Fairness of Explanationscited by 41
  17. 2019Can You Trust This Prediction? Auditing Pointwise Reliability After Learningcited by 28
  18. 2021How Well do Feature Visualizations Support Causal Understanding of CNN Activations?cited by 28
  19. 2016Identifying Unknown Unknowns in the Open World: Representations and Policies for Guided Explorationcited by 26
  20. 2021Understanding Instance-based Interpretability of Variational Auto-Encoderscited by 21
  21. 2022SEAL : Interactive Tool for Systematic Error Analysis and Labelingcited by 15
  22. 2016Debugging Machine Learning Taskscited by 9
  23. 2022Understanding out-of-distribution accuracies through quantifying difficulty of test samplescited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 3; 2026: 2.

7 papers.

  1. 2024What is Dataset Distillation Learning?cited by 13
  2. 2025Better Training Data Attribution via Better Inverse Hessian-Vector Productscited by 10
  3. 2024Towards User-Focused Research in Training Data Attribution for Human-Centered Explainable AIcited by 3
  4. 2026Measurement Under Selection: Decoy-Calibrated Failure Audits for Language Modelscited by 1
  5. 2025LLM generation novelty through the lens of semantic similaritycited by 0
  6. 2025GH-ESD: Grounded Hypothesis-Driven Error Slice Discovery for Instance-Level Vision Taskscited by 0
  7. 2026DeMix: Debugging Training Data with Mixed Data Error Types by Investigating Influence Vectorscited by 0