mlsanityData to 7 Oct 2026

Automated Text Mining of Experimental Methodologies from Biomedical Literature

arXiv 2404.13779 ↗First posted 21 Apr 2024cs.CL cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

Biomedical literature is a rapidly expanding field of science and technology. Classification of biomedical texts is an essential part of biomedicine research, especially in the field of biology. This work proposes the fine-tuned DistilBERT, a methodology-specific, pre-trained generative classification language model for mining biomedicine texts. The model has proven its effectiveness in linguistic understanding capabilities and has reduced the size of BERT models by 40% but by 60% faster. The main objective of this project is to improve the model and assess the performance of the model compared to the non-fine-tuned model. We used DistilBert as a support model and pre-trained on a corpus of 32,000 abstracts and complete text articles; our results were impressive and surpassed those of traditional literature classification methods by using RNN or LSTM. Our aim is to integrate this highly specialised and specific model into different research industries.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 20 of its 50 references that are in the map. First posted on arXiv 21 Apr 2024.

Builds on

20 of 50 references.

References · 20

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  4. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  5. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
  6. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,935
  7. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,184
  8. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,130
  9. 2014Distributed Representations of Sentences and Documentscited by 677
  10. 2017In-Datacenter Performance Analysis of a Tensor Processing Unitcited by 610
  11. 2022BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Miningcited by 531
  12. 2020Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlationcited by 420
8 more
  1. 2019Transfer Learning in Biomedical Natural Language Processing: An Evaluation of BERT and ELMo on Ten Benchmarking Datasetscited by 298
  2. 2014Reading Text in the Wild with Convolutional Neural Networkscited by 122
  3. 2016TextBoxes: A Fast Text Detector with a Single Deep Neural Networkcited by 119
  4. 2020A Comprehensive Survey on Word Representation Models: From Classical to State-Of-The-Art Word Representation Language Modelscited by 30
  5. 2022A sequence-to-sequence approach for document-level relation extractioncited by 25
  6. 2020Multi-label classification: do Hamming loss and subset accuracy really conflict with each other?cited by 14
  7. 2016NCBO Ontology Recommender 2.0: An Enhanced Approach for Biomedical Ontology Recommendationcited by 8
  8. 2022LitMC-BERT: transformer-based multi-label classification of biomedical literature with an application on COVID-19 literature curationcited by 6

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025QUAD-LLM-MLTC: Large Language Models Ensemble Learning for Healthcare Text Multi-Label Classificationcited by 1