mlsanityData to 14 Oct 2026

Modeling Disagreement in Automatic Data Labelling for Semi-Supervised Learning in Clinical Natural Language Processing

arXiv 2205.14761 ↗First posted 29 May 2022cs.LG cs.CL stat.MLcited by 0

Not on the map. Why?

Abstract (arXiv)

Computational models providing accurate estimates of their uncertainty are crucial for risk management associated with decision making in healthcare contexts. This is especially true since many state-of-the-art systems are trained using the data which has been labelled automatically (self-supervised mode) and tend to overfit. In this work, we investigate the quality of uncertainty estimates from a range of current state-of-the-art predictive models applied to the problem of observation detection in radiology reports. This problem remains understudied for Natural Language Processing in the healthcare domain. We demonstrate that Gaussian Processes (GPs) provide superior performance in quantifying the risks of 3 uncertainty labels based on the negative log predictive probability (NLPP) evaluation metric and mean maximum predicted confidence levels (MMPCL), whilst retaining strong predictive performance.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 15 of its 33 references that are in the map. First posted on arXiv 29 May 2022.

Builds on

15 of 33 references.

References · 15

  1. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  2. 2017On Calibration of Modern Neural Networkscited by 4,960
  3. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,926
  4. 2019CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparisoncited by 1,423
  5. 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,287
  6. 2019Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methodscited by 913
  7. 2019MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographscited by 597
  8. 2018Scalable and accurate deep learning for electronic health recordscited by 280
  9. 2017NegBio: a high-performance tool for negation and uncertainty detection in radiology reportscited by 79
  10. 2017Convolutional Gaussian Processescited by 44
  11. 2020On Power Laws in Deep Ensemblescited by 26
  12. 2015Improving the Gaussian Process Sparse Spectrum Approximation by Representing Uncertainty in Frequency Inputscited by 11
3 more
  1. 2019Towards calibrated and scalable uncertainty representations for neural networkscited by 10
  2. 2021Self-Supervised Detection of Contextual Synonyms in a Multi-Class Setting: Phenotype Annotation Use Casecited by 7
  3. 2018Clinical Assistant Diagnosis for Electronic Medical Record Based on Convolutional Neural Networkcited by 5

Built on by

Not cited yet.