mlsanityData to 7 Oct 2026

Domain-specific or Uncertainty-aware models: Does it really make a difference for biomedical text classification?

arXiv 2407.12626 ↗First posted 17 Jul 2024cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

The success of pretrained language models (PLMs) across a spate of use-cases has led to significant investment from the NLP community towards building domain-specific foundational models. On the other hand, in mission critical settings such as biomedical applications, other aspects also factor in-chief of which is a model's ability to produce reasonable estimates of its own uncertainty. In the present study, we discuss these two desiderata through the lens of how they shape the entropy of a model's output probability distribution. We find that domain specificity and uncertainty awareness can often be successfully combined, but the exact task at hand weighs in much more strongly.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 20 of its 37 references that are in the map. First posted on arXiv 17 Jul 2024.

Builds on

20 of 37 references.

References · 20

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  3. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  4. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,935
  5. 2019Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inferencecited by 864
  6. 2020A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challengescited by 813
  7. 2019Measuring Calibration in Deep Learningcited by 443
  8. 2019CamemBERT: a Tasty French Language Modelcited by 368
  9. 2020The Pitfalls of Simplicity Bias in Neural Networkscited by 349
  10. 2016Recommendations as Treatments: Debiasing Learning and Evaluationcited by 246
  11. 2018Lessons from Natural Language Inference in the Clinical Domaincited by 149
  12. 2022Uncertainty Quantification with Pre-trained Language Models: A Large-Scale Empirical Analysiscited by 104
8 more
  1. 2019DropConnect Is Effective in Modeling Uncertainty of Bayesian Deep Networkscited by 92
  2. 2018Quantifying Uncertainties in Natural Language Processing Taskscited by 79
  3. 2017Learning to select data for transfer learning with Bayesian Optimizationcited by 64
  4. 2023Uncertainty in Natural Language Processing: Sources, Quantification, and Applicationscited by 44
  5. 2022Pre-trained Language Models for the Legal Domain: A Case Study on Indian Lawcited by 39
  6. 2022Evaluating Unsupervised Text Classification: Zero-shot and Similarity-based Approachescited by 39
  7. 2017Neural Domain Adaptation for Biomedical Question Answeringcited by 30
  8. 2023FrenchMedMCQA: A French Multiple-Choice Question Answering Dataset for Medical domaincited by 25

Built on by

Not cited yet.