mlsanityData to 14 Oct 2026

Product-of-Experts Training Reduces Dataset Artifacts in Natural Language Inference

arXiv 2604.19069 ↗First posted 21 Apr 2026cs.CL cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Neural NLI models overfit dataset artifacts instead of truly reasoning. A hypothesis-only model gets 57.7% in SNLI, showing strong spurious correlations, and 38.6% of the baseline errors are the result of these artifacts. We propose Product-of-Experts (PoE) training, which downweights examples where biased models are overconfident. PoE nearly preserves accuracy (89.10% vs. 89.30%) while cutting bias reliance by 4.71% (bias agreement 49.85% to 45%). An ablation finds lambda = 1.5 that best balances debiasing and accuracy. Behavioral tests still reveal issues with negation and numerical reasoning.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 17 of its 24 references that are in the map. First posted on arXiv 21 Apr 2026.

Builds on

17 of 24 references.

References · 17

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  3. 2017On Calibration of Modern Neural Networkscited by 4,960
  4. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,448
  5. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,516
  6. 2015A large annotated corpus for learning natural language inferencecited by 1,847
  7. 2017Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semanticscited by 1,358
  8. 2020Beyond Accuracy: Behavioral Testing of NLP models with CheckListcited by 1,006
  9. 2019Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inferencecited by 876
  10. 2018Annotation Artifacts in Natural Language Inference Datacited by 671
  11. 2017Adversarial Examples for Evaluating Reading Comprehension Systemscited by 571
  12. 2017Synthetic and Natural Noise Both Break Neural Machine Translationcited by 306
5 more
  1. 2018Hypothesis Only Baselines in Natural Language Inferencecited by 295
  2. 2019Don't Take the Easy Way Out: Ensemble Based Methods for Avoiding Known Dataset Biasescited by 280
  3. 2020Towards Debiasing Sentence Representationscited by 171
  4. 2019Unlearn Dataset Bias in Natural Language Inference by Fitting the Residualcited by 119
  5. 2020Towards Debiasing NLU Models from Unknown Biasescited by 116

Built on by

Not cited yet.