mlsanityData to 7 Oct 2026

Large-scale investigation of weakly-supervised deep learning for the fine-grained semantic indexing of biomedical literature

arXiv 2301.09350 ↗First posted 23 Jan 2023cs.CL cs.DL cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Objective: Semantic indexing of biomedical literature is usually done at the level of MeSH descriptors with several related but distinct biomedical concepts often grouped together and treated as a single topic. This study proposes a new method for the automated refinement of subject annotations at the level of MeSH concepts. Methods: Lacking labelled data, we rely on weak supervision based on concept occurrence in the abstract of an article, which is also enhanced by dictionary-based heuristics. In addition, we investigate deep learning approaches, making design choices to tackle the particular challenges of this task. The new method is evaluated on a large-scale retrospective scenario, based on concepts that have been promoted to descriptors. Results: In our experiments concept occurrence was the strongest heuristic achieving a macro-F1 score of about 0.63 across several labels. The proposed method improved it further by more than 4pp. Conclusion: The results suggest that concept occurrence is a strong heuristic for refining the coarse-grained labels at the level of MeSH concepts and the proposed method improves it further.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 15 of its 29 references that are in the map. First posted on arXiv 23 Jan 2023.

Builds on

15 of 29 references.

References · 15

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2020Domain-Specific Language Model Pretraining for Biomedical Natural Language Processingcited by 996
  4. 2020Fine-Tuning Pretrained Language Models: Weight Initializations, Data Orders, and Early Stoppingcited by 377
  5. 2019Using Pre-Training Can Improve Model Robustness and Uncertaintycited by 373
  6. 2017Snorkel: Rapid Training Data Creation with Weak Supervisioncited by 370
  7. 2016Data Programming: Creating Large Training Sets, Quicklycited by 246
  8. 2020Distribution-Balanced Loss for Multi-Label Classification in Long-Tailed Datasetscited by 122
  9. 2021Balancing Methods for Multi-label Text Classification with Long-Tailed Class Distributioncited by 34
  10. 2021Memorisation versus Generalisation in Pre-trained Language Modelscited by 32
  11. 2021Coarse2Fine: Fine-grained Text Classification on Coarsely-grained Annotated Datacited by 13
  12. 2022Overview of BioASQ 2022: The tenth BioASQ challenge on Large-Scale Biomedical Semantic Indexing and Question Answeringcited by 9
3 more
  1. 2021Classifying Scientific Publications with BERT -- Is Self-Attention a Feature Selection Method?cited by 3
  2. 2020Beyond MeSH: Fine-Grained Semantic Indexing of Biomedical Literature based on Weak Supervisioncited by 1
  3. 2021What is all this new MeSH about? Exploring the semantic provenance of new descriptors in the MeSH thesauruscited by 1

Built on by

Not cited yet.