mlsanityData to 14 Oct 2026

Benchmarking a foundation LLM on its ability to re-label structure names in accordance with the AAPM TG-263 report

arXiv 2310.03874 ↗First posted 5 Oct 2023physics.med-ph cs.CLcited by 5

Not on the map. Why?

Abstract (arXiv)

Purpose: To introduce the concept of using large language models (LLMs) to re-label structure names in accordance with the American Association of Physicists in Medicine (AAPM) Task Group (TG)-263 standard, and to establish a benchmark for future studies to reference. Methods and Materials: The Generative Pre-trained Transformer (GPT)-4 application programming interface (API) was implemented as a Digital Imaging and Communications in Medicine (DICOM) storage server, which upon receiving a structure set DICOM file, prompts GPT-4 to re-label the structure names of both target volumes and normal tissues according to the AAPM TG-263. Three disease sites, prostate, head and neck, and thorax were selected for evaluation. For each disease site category, 150 patients were randomly selected for manually tuning the instructions prompt (in batches of 50) and 50 patients were randomly selected for evaluation. Structure names that were considered were those that were most likely to be relevant for studies utilizing structure contours for many patients. Results: The overall re-labeling accuracy of both target volumes and normal tissues for prostate, head and neck, and thorax cases was 96.0%, 98.5%, and 96.9% respectively. Re-labeling of target volumes was less accurate on average except for prostate - 100%, 93.1%, and 91.1% respectively. Conclusions: Given the accuracy of GPT-4 in re-labeling structure names of both target volumes and normal tissues as presented in this work, LLMs are poised to be the preferred method for standardizing structure names in radiation oncology, especially considering the rapid advancements in LLM capabilities that are likely to continue.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 9 of its 72 references that are in the map. First posted on arXiv 5 Oct 2023.

Builds on

9 of 72 references.

References · 9

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  2. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  3. 2023DeID-GPT: Zero-shot Medical Text De-Identification by GPT-4cited by 130
  4. 2023Evaluating Large Language Models on a Highly-specialized Topic, Radiation Oncology Physicscited by 49
  5. 2023Exploring the Trade-Offs: Unified Large Language Models vs Local Fine-Tuned Models for Highly-Specific Radiology NLI Taskcited by 32
  6. 2023Mask-guided BERT for Few Shot Text Classificationcited by 25
  7. 2023Segment Anything Model (SAM) for Radiation Oncologycited by 24
  8. 2023RadOnc-GPT: A Large Language Model for Radiation Oncologycited by 18
  9. 2021PSA-Net: Deep Learning based Physician Style-Aware Segmentation Network for Post-Operative Prostate Cancer Clinical Target Volumecited by 9

Built on by

Cited by year

Citing papers per year: 2023: 3; 2024: 1; 2025: 1.

5 papers.

  1. 2024Revolutionizing Finance with LLMs: An Overview of Applications and Insightscited by 90
  2. 2023Holistic Evaluation of GPT-4V for Biomedical Imagingcited by 19
  3. 2023Ophtha-LLaMA2: A Large Language Model for Ophthalmologycited by 9
  4. 2023Evaluating multiple large language models in pediatric ophthalmologycited by 4
  5. 2025Large Language Models for Bioinformaticscited by 3