mlsanityData to 14 Oct 2026

Surpassing GPT-4 Medical Coding with a Two-Stage Approach

arXiv 2311.13735 ↗First posted 22 Nov 2023cs.CLcited by 6

Not on the map. Why?

Abstract (arXiv)

Recent advances in large language models (LLMs) show potential for clinical applications, such as clinical decision support and trial recommendations. However, the GPT-4 LLM predicts an excessive number of ICD codes for medical coding tasks, leading to high recall but low precision. To tackle this challenge, we introduce LLM-codex, a two-stage approach to predict ICD codes that first generates evidence proposals using an LLM and then employs an LSTM-based verification stage. The LSTM learns from both the LLM's high recall and human expert's high precision, using a custom loss function. Our model is the only approach that simultaneously achieves state-of-the-art results in medical coding accuracy, accuracy on rare codes, and sentence-level evidence identification to support coding decisions without training on human-annotated evidence according to experiments on the MIMIC dataset.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 39 of its 67 references that are in the map. First posted on arXiv 22 Nov 2023.

Builds on

39 of 67 references.

References · 39

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  3. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,392
  4. 2019Attention is not Explanationcited by 929
  5. 2021MetaICL: Learning to Learn In Contextcited by 454
  6. 2022Large Language Models are Better Reasoners with Self-Verificationcited by 307
  7. 2023MedAlpaca -- An Open-Source Collection of Medical Conversational AI Models and Training Datacited by 286
  8. 2018Explainable Prediction of Medical Codes from Clinical Textcited by 234
  9. 2022Large Language Models are Few-Shot Clinical Information Extractorscited by 207
  10. 2017Explainable Artificial Intelligence: Understanding, Visualizing and Interpreting Deep Learning Modelscited by 206
  11. 2017Simple and Effective Multi-Paragraph Reading Comprehensioncited by 120
  12. 2022Thinking about GPT-3 In-Context Learning for Biomedical IE? Think Againcited by 83
27 more
  1. 2019A Discrete Hard EM Approach for Weakly Supervised Question Answeringcited by 78
  2. 2022Improving In-Context Few-Shot Learning via Self-Supervised Trainingcited by 57
  3. 2022Code Synonyms Do Matter: Multiple Synonyms Matching Network for Automatic ICD Codingcited by 53
  4. 2023Self-Verification Improves Few-Shot Clinical Information Extractioncited by 52
  5. 2022PLM-ICD: Automatic ICD Coding with Pretrained Language Modelscited by 49
  6. 2022Knowledge Injected Prompt Based Fine-tuning for Multi-label Few-shot ICD Codingcited by 40
  7. 2020Explainable Automated Coding of Clinical Notes using Hierarchical Label-wise Attention Networks and Label Embedding Initialisationcited by 38
  8. 2021Perturbing Inputs for Fragile Interpretations in Deep Natural Language Processingcited by 30
  9. 2022Clinical Prompt Learning with Frozen Language Modelscited by 19
  10. 2023Mimic-IV-ICD: A new benchmark for eXtreme MultiLabel Classificationcited by 19
  11. 2023MDACE: MIMIC Documents Annotated with Code Evidencecited by 19
  12. 2023Identifying and Extracting Rare Disease Phenotypes with Large Language Modelscited by 18
  13. 2022Multi-label Few-shot ICD Coding as Autoregressive Generation with Promptcited by 17
  14. 2020Why do you think that? Exploring Faithful Sentence-Level Rationales Without Supervisioncited by 15
  15. 2023CHiLL: Zero-shot Custom Interpretable Feature Extraction from Clinical Notes with Large Language Modelscited by 15
  16. 2022Fooling Explanations in Text Classifierscited by 13
  17. 2022ICDBigBird: A Contextual Embedding Model for ICD Code Classificationcited by 10
  18. 2023A Two-Stage Decoder for Efficient ICD Codingcited by 9
  19. 2022Can Current Explainability Help Provide References in Clinical Notes to Support Humans Annotate Medical Codes?cited by 7
  20. 2022Automated Identification of Eviction Status from Electronic Health Record Notescited by 6
  21. 2021Rare Disease Identification from Clinical Notes with Ontologies and Weak Supervisioncited by 5
  22. 2022Associations Between Natural Language Processing (NLP) Enriched Social Determinants of Health and Suicide Death among US Veteranscited by 5
  23. 2020Dynamically Extracting Outcome-Specific Problem Lists from Clinical Notes with Guided Multi-Headed Attentioncited by 4
  24. 2022Extend and Explain: Interpreting Very Long Language Modelscited by 4
  25. 2023ODD: A Benchmark Dataset for the Natural Language Processing based Opioid Related Aberrant Behavior Detectioncited by 3
  26. 2022OSLAT: Open Set Label Attention Transformer for Medical Entity Retrieval and Span Extractioncited by 1
  27. 2023DARE: Towards Robust Text Explanations in Biomedical and Healthcare Applicationscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 1; 2026: 2.

6 papers.

  1. 2025Reinforcement Learning for Out-of-Distribution Reasoning in LLMs: An Empirical Study on Diagnosis-Related Group Codingcited by 7
  2. 2024DILA: Dictionary Label Attention for Mechanistic Interpretability in High-dimensional Multi-label Medical Coding Predictioncited by 4
  3. 2024Stochastic Parrots or ICU Experts? Large Language Models in Critical Care Medicine: A Scoping Reviewcited by 0
  4. 2024Don't Pay Attention, PLANT It: Pretraining Attention via Learning-to-Rankcited by 0
  5. 2026Automated ICD Classification of Psychiatric Diagnoses: From Classical NLP to Large Language Modelscited by 0
  6. 2026LearnActCoder: Role-Aware Error Memory for Adaptive Clinical Coding Agentscited by 0