# Pre-trained Language Models and Few-shot Learning for Medical Entity Extraction

arXiv: https://arxiv.org/abs/2504.04385 · HTML: https://mlsanity.com/paper/2504.04385/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 27 references that are in the map.
- First posted on arXiv 6 Apr 2025.

## Builds on (most cited first, 10 of 10)

- [Survival Prediction Across Diverse Cancer Types Using Neural Networks](https://mlsanity.com/paper/2404.08713/) (2024)
- [Self-Supervised Graph Neural Networks for Enhanced Feature Extraction in Heterogeneous Information Networks](https://mlsanity.com/paper/2410.17617/) (2024)
- [Contrastive Learning for Cold Start Recommendation with Adaptive Feature Fusion](https://mlsanity.com/paper/2502.03664/) (2025)
- [Object Detection for Medical Image Analysis: Insights from the RT-DETR Model](https://mlsanity.com/paper/2501.16469/) (2025)
- [Optimized Unet with Attention Mechanism for Multi-Scale Semantic Segmentation](https://mlsanity.com/paper/2502.03813/) (2025)
- [Context-Aware Rule Mining Using a Dynamic Transformer-Based Framework](https://mlsanity.com/paper/2503.11125/) (2025)
- [Comprehensive Overview of Named Entity Recognition: Models, Domain-Specific Applications and Challenges](https://mlsanity.com/paper/2309.14084/) (2023)
- [Multi-Scale Transformer Architecture for Accurate Medical Image Classification](https://mlsanity.com/paper/2502.06243/) (2025)
- [A Hybrid CNN-Transformer Model for Heart Disease Prediction Using Life History Data](https://mlsanity.com/paper/2503.02124/) (2025)
- [Adaptive Transformer Attention and Multi-Scale Fusion for Spine 3D Segmentation](https://mlsanity.com/paper/2503.12853/) (2025)

## Built on by (most cited first, 4 of 4)

- [Clinical NLP with Attention-Based Deep Learning for Multi-Disease Prediction](https://mlsanity.com/paper/2507.01437/) (2025)
- [State-Aware IoT Scheduling Using Deep Q-Networks and Edge-Based Coordination](https://mlsanity.com/paper/2504.15577/) (2025)
- [Intelligent Task Scheduling for Microservices via A3C-Based Reinforcement Learning](https://mlsanity.com/paper/2505.00299/) (2025)
- [Topology-Aware Graph Reinforcement Learning for Dynamic Routing in Cloud Networks](https://mlsanity.com/paper/2509.04973/) (2025)

## Abstract (arXiv)

This study proposes a medical entity extraction method based on Transformer to enhance the information extraction capability of medical literature. Considering the professionalism and complexity of medical texts, we compare the performance of different pre-trained language models (BERT, BioBERT, PubMedBERT, ClinicalBERT) in medical entity extraction tasks. Experimental results show that PubMedBERT achieves the best performance (F1-score = 88.8%), indicating that a language model pre-trained on biomedical literature is more effective in the medical domain. In addition, we analyze the impact of different entity extraction methods (CRF, Span-based, Seq2Seq) and find that the Span-based approach performs best in medical entity extraction tasks (F1-score = 88.6%). It demonstrates superior accuracy in identifying entity boundaries. In low-resource scenarios, we further explore the application of Few-shot Learning in medical entity extraction. Experimental results show that even with only 10-shot training samples, the model achieves an F1-score of 79.1%, verifying the effectiveness of Few-shot Learning under limited data conditions. This study confirms that the combination of pre-trained language models and Few-shot Learning can enhance the accuracy of medical entity extraction. Future research can integrate knowledge graphs and active learning strategies to improve the model's generalization and stability, providing a more effective solution for medical NLP research. Keywords- Natural Language Processing, medical named entity recognition, pre-trained language model, Few-shot Learning, information extraction, deep learning