mlsanityData to 7 Oct 2026

An Empirical Study of Multi-Task Learning on BERT for Biomedical Text Mining

arXiv 2005.02799 ↗First posted 6 May 2020cs.CLcited by 35

Not on the map. Why?

Abstract (arXiv)

Multi-task learning (MTL) has achieved remarkable success in natural language processing applications. In this work, we study a multi-task learning model with multiple decoders on varieties of biomedical and clinical natural language processing tasks such as text similarity, relation extraction, named entity recognition, and text inference. Our empirical results demonstrate that the MTL fine-tuned models outperform state-of-the-art transformer models (e.g., BERT and its variants) by 2.0% and 1.3% in biomedical and clinical domains, respectively. Pairwise MTL further demonstrates more details about which tasks can improve or decrease others. This is particularly helpful in the context that researchers are in the hassle of choosing a suitable model for new problems. The code and models are publicly available at https://github.com/ncbi-nlp/bluebert

Key facts. Cited by 35 papers in the map (data through 7 Oct 2026). First posted on arXiv 6 May 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 5; 2022: 7; 2023: 9; 2024: 6; 2025: 7; 2026: 1.

35 papers.

Sort:

By area:

  1. 2021AMMUS : A Survey of Transformer-based Pretrained Models in Natural Language Processingcited by 131
  2. 2024A Review of Large Language Models and Autonomous Agents in Chemistrycited by 96
  3. 2022BioBART: Pretraining and Evaluation of A Biomedical Generative Language Modelcited by 83
  4. 2024Scientific Large Language Models: A Survey on Biological & Chemical Domainscited by 80
  5. 2021AMMU : A Survey of Transformer-based Biomedical Pretrained Language Modelscited by 32
  6. 2022When to Use Multi-Task Learning vs Intermediate Fine-Tuning for Pre-Trained Encoder Transfer Learningcited by 17
  7. 2025A Large-Language Model Framework for Relative Timeline Extraction from PubMed Case Reportscited by 10
  8. 2022Better Few-Shot Relation Extraction with Label Prompt Dropoutcited by 8
  9. 2023BioREx: Improving Biomedical Relation Extraction by Leveraging Heterogeneous Datasetscited by 8
  10. 2023Biomedical Language Models are Robust to Sub-optimal Tokenizationcited by 8
  11. 2021How Do Your Biomedical Named Entity Recognition Models Generalize to Novel Entities?cited by 7
  12. 2023Collaborative Synthesis of Patient Records through Multi-Visit Health State Inferencecited by 6
23 more
  1. 2021Transformer-based Multi-task Learning for Disaster Tweet Categorisationcited by 5
  2. 2023When Does Aggregating Multiple Skills with Multi-Task Learning Work? A Case Study in Financial NLPcited by 5
  3. 2025Forecasting Clinical Risk from Textual Time Series: Structuring Narratives for Temporal AI in Healthcarecited by 5
  4. 2024Identifying Task Groupings for Multi-Task Learning Using Pointwise V-Usable Informationcited by 4
  5. 2024Generative Design of Functional Metal Complexes Utilizing the Internal Knowledge of Large Language Modelscited by 4
  6. 2023STG-MTL: Scalable Task Grouping for Multi-Task Learning Using Data Mapcited by 3
  7. 2023UniDCP: Unifying Multiple Medical Vision-language Tasks via Dynamic Cross-modal Learnable Promptscited by 3
  8. 2025PMOA-TTS: Introducing the PubMed Open Access Textual Times Series Corpuscited by 3
  9. 2022Efficient Extraction of Pathologies from C-Spine Radiology Reports using Multi-Task Learningcited by 2
  10. 2022Do Language Models Understand Measurements?cited by 2
  11. 2023A Cross-institutional Evaluation on Breast Cancer Phenotyping NLP Algorithms on Electronic Health Recordscited by 2
  12. 2025Retrieval-Augmented Framework for LLM-Based Clinical Decision Supportcited by 2
  13. 2025HARMON-E: Hierarchical Agentic Reasoning for Multimodal Oncology Notes to Extract Structured Datacited by 2
  14. 2022Explaining the Effectiveness of Multi-Task Learning for Efficient Knowledge Extraction from Spine MRI Reportscited by 1
  15. 2023Bent & Broken Bicycles: Leveraging synthetic data for damaged object re-identificationcited by 1
  16. 2023Into the Single Cell Multiverse: an End-to-End Dataset for Procedural Knowledge Extraction in Biomedical Textscited by 1
  17. 2025Coarse-to-Fine Personalized LLM Impressions for Streamlined Radiology Reportscited by 1
  18. 2021An Open-Source Dataset and A Multi-Task Model for Malay Named Entity Recognitioncited by 0
  19. 2022Don't Waste Data: Transfer Learning to Leverage All Data for Machine-Learnt Climate Model Emulationcited by 0
  20. 2024Improving Pre-trained Language Model Sensitivity via Mask Specific losses: A case study on Biomedical NERcited by 0
  21. 2024Multitask Fine-Tuning and Generative Adversarial Learning for Improved Auxiliary Classificationcited by 0
  22. 2025Task-Specific Sparse Feature Masks for Molecular Toxicity Prediction with Chemical Language Modelscited by 0
  23. 2026A Semi-Automated Annotation Workflow for Paediatric Histopathology Reports Using Small Language Modelscited by 0