mlsanityData to 14 Oct 2026

A Semi-Automated Annotation Workflow for Paediatric Histopathology Reports Using Small Language Models

arXiv 2604.04168 ↗First posted 5 Apr 2026cs.CL cs.IRcited by 0

Not on the map. Why?

Abstract (arXiv)

Electronic Patient Record (EPR) systems contain valuable clinical information, but much of it is trapped in unstructured text, limiting its use for research and decision-making. Large language models can extract such information but require substantial computational resources to run locally, and sending sensitive clinical data to cloud-based services, even when deidentified, raises significant patient privacy concerns. In this study, we develop a resource-efficient semi-automated annotation workflow using small language models (SLMs) to extract structured information from unstructured EPR data, focusing on paediatric histopathology reports. As a proof-of-concept, we apply the workflow to paediatric renal biopsy reports, a domain chosen for its constrained diagnostic scope and well-defined underlying biology. We develop the workflow iteratively with clinical oversight across three meetings, manually annotating 400 reports from a dataset of 2,111 at Great Ormond Street Hospital as a gold standard, while developing an automated information extraction approach using SLMs. We frame extraction as a Question-Answering task grounded by clinician-guided entity guidelines and few-shot examples, evaluating five instruction-tuned SLMs with a disagreement modelling framework to prioritise reports for clinical review. Gemma 2 2B achieves the highest accuracy at 84.3%, outperforming off-the-shelf models including spaCy (74.3%), BioBERT-SQuAD (62.3%), RoBERTa-SQuAD (59.7%), and GLiNER (60.2%). Entity guidelines improved performance by 7-19% over the zero-shot baseline, and few-shot examples by 6-38%, though their benefits do not compound when combined. These results demonstrate that SLMs can extract structured information from specialised clinical domains on CPU-only infrastructure with minimal clinician involvement. Our code is available at https://github.com/gosh-dre/nlp_renal_biopsy.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 24 of its 45 references that are in the map. First posted on arXiv 5 Apr 2026.

Builds on

24 of 45 references.

References · 24

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  4. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,947
  5. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,601
  6. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,554
  7. 2019Publicly Available Clinical BERT Embeddingscited by 1,002
  8. 2024LLM Evaluators Recognize and Favor Their Own Generationscited by 686
  9. 2019ScispaCy: Fast and Robust Models for Biomedical Natural Language Processingcited by 244
  10. 2024Large Language Models for Data Annotation and Synthesis: A Surveycited by 205
  11. 2023GPT-NER: Named Entity Recognition via Large Language Modelscited by 160
  12. 2020Multi-domain Clinical Natural Language Processing with MedCAT: the Medical Concept Annotation Toolkitcited by 66
12 more
  1. 2021Launching into clinical space with medspaCy: a new clinical text processing toolkit in Pythoncited by 40
  2. 2020An Empirical Study of Multi-Task Learning on BERT for Biomedical Text Miningcited by 35
  3. 2023PromptNER: Prompting For Named Entity Recognitioncited by 35
  4. 2024NuNER: Entity Recognition Encoder Pre-training via LLM-Annotated Datacited by 34
  5. 2024PediatricsGPT: Large Language Models as Chinese Medical Assistants for Pediatric Applicationscited by 16
  6. 2023FreeAL: Towards Human-Free Active Learning in the Era of Large Language Modelscited by 14
  7. 2022PathologyBERT -- Pre-trained Vs. A New Transformer Language Model for Pathology Domaincited by 12
  8. 2024On-the-fly Definition Augmentation of LLMs for Biomedical NERcited by 9
  9. 2023Local Large Language Models for Complex Structured Medical Taskscited by 7
  10. 2023Test-Time Self-Adaptive Small Language Models for Question Answeringcited by 7
  11. 2023Developing a Named Entity Recognition Dataset for Tagalogcited by 7
  12. 2024MedDoc-Bot: A Chat Tool for Comparative Analysis of Large Language Models in the Context of the Pediatric Hypertension Guidelinecited by 2

Built on by

Not cited yet.