Multi-Task LLM with LoRA Fine-Tuning for Automated Cancer Staging and Biomarker Extraction
Not on the map. Why?
Abstract (arXiv)
Pathology reports serve as the definitive record for breast cancer staging, yet their unstructured format impedes large-scale data curation. While Large Language Models (LLMs) offer semantic reasoning, their deployment is often limited by high computational costs and hallucination risks. This study introduces a parameter-efficient, multi-task framework for automating the extraction of Tumor-Node-Metastasis (TNM) staging, histologic grade, and biomarkers. We fine-tune a Llama-3-8B-Instruct encoder using Low-Rank Adaptation (LoRA) on a curated, expert-verified dataset of 10,677 reports. Unlike generative approaches, our architecture utilizes parallel classification heads to enforce consistent schema adherence. Experimental results demonstrate that the model achieves a Macro F1 score of 0.976, successfully resolving complex contextual ambiguities and heterogeneous reporting formats that challenge traditional extraction methods including rule-based natural language processing (NLP) pipelines, zero-shot LLMs, and single-task LLM baselines. The proposed adapter-efficient, multi-task architecture enables reliable, scalable pathology-derived cancer staging and biomarker profiling, with the potential to enhance clinical decision support and accelerate data-driven oncology research.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 21 of its 46 references that are in the map. First posted on arXiv 14 Apr 2026.
Builds on
21 of 46 references.
References · 21
- 2017Attention Is All You Needcited by 59,932
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2024The Llama 3 Herd of Modelscited by 14,192
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
- 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
- 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
- 2023Efficient Streaming Language Models with Attention Sinkscited by 2,115
- 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,935
- 2019Publicly Available Clinical BERT Embeddingscited by 995
- 2022Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learningcited by 886
- 2017A Survey on Multi-Task Learningcited by 838
9 more
- 2017An Overview of Multi-Task Learning in Deep Neural Networkscited by 791
- 2022Large Language Models are Few-Shot Clinical Information Extractorscited by 206
- 2023Scaling Down to Scale Up: A Guide to Parameter-Efficient Fine-Tuningcited by 171
- 2022Hierarchical Deep Learning with Generative Adversarial Network for Automatic Cardiac Diagnosis from ECG Signalscited by 15
- 2025Large Language Models Encode Semantics and Alignment in Linearly Separable Representationscited by 14
- 2023Automated Identication of Atrial Fibrillation from Single-lead ECGs Using Multi-branching ResNetcited by 9
- 2024BA-LoRA: Bias-Alleviating Low-Rank Adaptation to Mitigate Catastrophic Inheritance in Large Language Modelscited by 7
- 2024MUSE-Net: Missingness-aware mUlti-branching Self-attention Encoder for Irregular Longitudinal Electronic Health Recordscited by 4
- 2025Transformer with Koopman-Enhanced Graph Convolutional Network for Spatiotemporal Dynamics Forecastingcited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Knowledge Elicitation with Large Language Models for Interpretable Cancer Stage Identification from Pathology Reports
- 2026SIFTING: A Novel LLM-Based Framework for Structured and Transparent Information Extraction from Clinical Free-Text Reports, with Application to Tumor Staging in Lung Cancer
- 2024Beyond Self-Consistency: Ensemble Reasoning Boosts Consistency and Accuracy of LLMs in Cancer Staging
- 2025Towards Scalable and Cross-Lingual Specialist Language Models for Oncology
- 2026Prompt, Plan, Extract: Zero-Shot Agentic LLMs Workflows for Lung Pathology Extraction from Clinical Narratives
- 2024Language Models and Retrieval Augmented Generation for Automated Structured Data Extraction from Diagnostic Reports
- 2025AMRG: Extend Vision Language Models for Automatic Mammography Report Generation
- 2026An end-to-end-trained vision-language model for native-language prostate pathology report generation