mlsanityData to 14 Oct 2026

Unlocking Historical Clinical Trial Data with ALIGN: A Compositional Large Language Model System for Medical Coding

arXiv 2411.13163 ↗First posted 20 Nov 2024cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

The reuse of historical clinical trial data has significant potential to accelerate medical research and drug development. However, interoperability challenges, particularly with missing medical codes, hinders effective data integration across studies. While Large Language Models (LLMs) offer a promising solution for automated coding without labeled data, current approaches face challenges on complex coding tasks. We introduce ALIGN, a novel compositional LLM-based system for automated, zero-shot medical coding. ALIGN follows a three-step process: (1) diverse candidate code generation; (2) self-evaluation of codes and (3) confidence scoring and uncertainty estimation enabling human deferral to ensure reliability. We evaluate ALIGN on harmonizing medication terms into Anatomical Therapeutic Chemical (ATC) and medical history terms into Medical Dictionary for Regulatory Activities (MedDRA) codes extracted from 22 immunology trials. ALIGN outperformed the LLM baselines, while also providing capabilities for trustworthy deployment. For MedDRA coding, ALIGN achieved high accuracy across all levels, matching RAG and excelling at the most specific levels (87-90% for HLGT). For ATC coding, ALIGN demonstrated superior performance, particularly at lower hierarchy levels (ATC Level 4), with 72-73% overall accuracy and 86-89% accuracy for common medications, outperforming baselines by 7-22%. ALIGN's uncertainty-based deferral improved accuracy by 17% to 90% accuracy with 30% deferral, notably enhancing performance on uncommon medications. ALIGN achieves this cost-efficiently at 0.0007 and 0.02 per code for GPT-4o-mini and GPT-4o, reducing barriers to clinical adoption. ALIGN advances automated medical coding for clinical trial data, contributing to enhanced data interoperability and reusability, positioning it as a promising tool to improve clinical research and accelerate drug development.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 16 of its 37 references that are in the map. First posted on arXiv 20 Nov 2024.

Builds on

16 of 37 references.

References · 16

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  3. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  4. 2022Language Models (Mostly) Know What They Knowcited by 1,856
  5. 2015A large annotated corpus for learning natural language inferencecited by 1,847
  6. 2022Large Language Models Encode Clinical Knowledgecited by 1,830
  7. 2023Large Language Models as Optimizerscited by 864
  8. 2023Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedbackcited by 812
  9. 2024BM25S: Orders of magnitude faster lexical search via eager sparse scoringcited by 88
  10. 2023Self-Evaluation Improves Selective Generation in Large Language Modelscited by 76
  11. 2023Curated LLM: Synergy of LLMs and Data Curation for tabular augmentation in low-data regimescited by 52
  12. 2024Can Large Language Models abstract Medical Coded Language?cited by 20
4 more
  1. 2022DC-Check: A Data-Centric AI checklist to guide the development of reliable machine learning systemscited by 14
  2. 2024Are Machines Better at Complex Reasoning? Unveiling Human-Machine Inference Gaps in Entailment Verificationcited by 13
  3. 2024Multi-stage Retrieve and Re-rank Model for Automatic Medical Coding Recommendationcited by 8
  4. 2024Large language models are good medical coders, if provided with toolscited by 8

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Training a Large Language Model for Medical Coding Using Privacy-Preserving Synthetic Clinical Data
  2. 2026KREL: Automatic Medical Coding via Knowledge-Guided Reasoning over Clinical Evidence with LLMs
  3. 2025Multimodal Medical Code Tokenizer
  4. 2025Developing Large Language Models for Clinical Research Using One Million Clinical Trials
  5. 2026Exploring Post-Training Alignment of Small Language Models for Biomedical Data-to-Text Generation: A Case Study of Medication Leaflet
  6. 2026Symphony for Medical Coding: A Next-Generation Agentic System for Scalable and Explainable Medical Coding
  7. 2025DualAlign: Generating Clinically Grounded Synthetic Data
  8. 2026When Medical Safety Alignment Fails: A Benchmark for Evaluating LLMs on High-Risk Medical Queries