mlsanityData to 14 Oct 2026

Confidence Calibration in Large Language Model-Based Entity Matching

arXiv 2509.19557 ↗First posted 23 Sep 2025cs.CL cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

This research aims to explore the intersection of Large Language Models and confidence calibration in Entity Matching. To this end, we perform an empirical study to compare baseline RoBERTa confidences for an Entity Matching task against confidences that are calibrated using Temperature Scaling, Monte Carlo Dropout and Ensembles. We use the Abt-Buy, DBLP-ACM, iTunes-Amazon and Company datasets. The findings indicate that the proposed modified RoBERTa model exhibits a slight overconfidence, with Expected Calibration Error scores ranging from 0.0043 to 0.0552 across datasets. We find that this overconfidence can be mitigated using Temperature Scaling, reducing Expected Calibration Error scores by up to 23.83%.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 20 of its 44 references that are in the map. First posted on arXiv 23 Sep 2025.

Builds on

20 of 44 references.

References · 20

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  4. 2020Language Models are Few-Shot Learnerscited by 32,419
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  6. 2017On Calibration of Modern Neural Networkscited by 4,960
  7. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,448
  8. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
  9. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,811
  10. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,385
  11. 2020How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answeringcited by 445
  12. 2020Calibrating Deep Neural Networks using Focal Losscited by 357
8 more
  1. 2020Calibration of Pre-trained Transformerscited by 315
  2. 2019Verified Uncertainty Calibrationcited by 309
  3. 2022Can Foundation Models Wrangle Your Data?cited by 175
  4. 2020Deep Entity Matching with Pre-Trained Language Modelscited by 144
  5. 2020Uncertainty Quantification and Deep Ensemblescited by 139
  6. 2022Uncertainty Quantification with Pre-trained Language Models: A Large-Scale Empirical Analysiscited by 104
  7. 2018Quantifying Uncertainties in Natural Language Processing Taskscited by 79
  8. 2022Supervised Contrastive Learning for Product Matchingcited by 24

Built on by

Not cited yet.