mlsanityData to 7 Oct 2026

PLM-ICD: Automatic ICD Coding with Pretrained Language Models

arXiv 2207.05289 ↗First posted 12 Jul 2022cs.CL cs.AIcited by 49

Not on the map. Why?

Abstract (arXiv)

Automatically classifying electronic health records (EHRs) into diagnostic codes has been challenging to the NLP community. State-of-the-art methods treated this problem as a multilabel classification problem and proposed various architectures to model this problem. However, these systems did not leverage the superb performance of pretrained language models, which achieved superb performance on natural language understanding tasks. Prior work has shown that pretrained language models underperformed on this task with the regular finetuning scheme. Therefore, this paper aims at analyzing the causes of the underperformance and developing a framework for automatic ICD coding with pretrained language models. We spotted three main issues through the experiments: 1) large label space, 2) long input sequences, and 3) domain mismatch between pretraining and fine-tuning. We propose PLMICD, a framework that tackles the challenges with various strategies. The experimental results show that our proposed framework can overcome the challenges and achieves state-of-the-art performance in terms of multiple metrics on the benchmark MIMIC data. The source code is available at https://github.com/MiuLab/PLM-ICD

Key facts. Cited by 49 papers in the map (data through 7 Oct 2026). Builds on 25 of its 41 references that are in the map. First posted on arXiv 12 Jul 2022.

Builds on

25 of 41 references.

References · 25

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  5. 2020Longformer: The Long-Document Transformercited by 3,362
  6. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
  7. 2018Deep contextualized word representationscited by 2,285
  8. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,935
  9. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,548
  10. 2015Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Bookscited by 1,041
  11. 2020Domain-Specific Language Model Pretraining for Biomedical Natural Language Processingcited by 996
  12. 2019Publicly Available Clinical BERT Embeddingscited by 995
13 more
  1. 2019How to Fine-Tune BERT for Text Classification?cited by 409
  2. 2018Explainable Prediction of Medical Codes from Clinical Textcited by 231
  3. 2019Neural Legal Judgment Prediction in Englishcited by 208
  4. 2015Doctor AI: Predicting Clinical Events via Recurrent Neural Networkscited by 205
  5. 2018AttentionXML: Label Tree-based Attention-Aware Deep Model for High-Performance Extreme Multi-Label Text Classificationcited by 101
  6. 2019ICD Coding from Clinical Text Using Multi-Filter Residual Convolutional Neural Networkcited by 73
  7. 2020An Empirical Study on Large-Scale Multi-Label Text Classification Including Few and Zero-Shot Labelscited by 44
  8. 2017Multi-Label Classification of Patient Notes a Case Study on ICD Code Assignmentcited by 37
  9. 2020BERT-XML: Large Scale Automated ICD Coding Using BERT Pretrainingcited by 37
  10. 2017Towards Automated ICD Coding Using Deep Learningcited by 35
  11. 2020Inferring Degrees from Incomplete Networks and Nonlinear Dynamicscited by 33
  12. 2021Towards BERT-based Automatic ICD Coding: Limitations and Opportunitiescited by 25
  13. 2021Read, Attend, and Code: Pushing the Limits of Medical Codes Prediction from Clinical Notes by Machinescited by 23

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 13; 2024: 11; 2025: 10; 2026: 13.

49 papers.

Sort:

By area:

  1. 2023A Survey of Large Language Models in Medicine: Progress, Application, and Challengecited by 123
  2. 2023Automated Medical Coding on MIMIC-III and MIMIC-IV: A Critical Review and Replicability Studycited by 48
  3. 2022Knowledge Injected Prompt Based Fine-tuning for Multi-label Few-shot ICD Codingcited by 40
  4. 2024Large Language Models in Drug Discovery and Development: From Disease Mechanisms to Clinical Trialscited by 24
  5. 2023Mimic-IV-ICD: A new benchmark for eXtreme MultiLabel Classificationcited by 19
  6. 2023MDACE: MIMIC Documents Annotated with Code Evidencecited by 19
  7. 2023Automated clinical coding using off-the-shelf large language modelscited by 14
  8. 2024Can GPT-3.5 Generate and Code Discharge Summaries?cited by 12
  9. 2023Knowledge Graphs for the Life Sciences: Recent Developments, Challenges and Opportunitiescited by 10
  10. 2024Almanac Copilot: Towards Autonomous Electronic Health Record Navigationcited by 9
  11. 2023A Two-Stage Decoder for Efficient ICD Codingcited by 8
  12. 2023Modelling Temporal Document Sequences for Clinical ICD Codingcited by 7
37 more
  1. 2023Surpassing GPT-4 Medical Coding with a Two-Stage Approachcited by 6
  2. 2022Automated ICD Coding using Extreme Multi-label Long Text Transformer-based Modelscited by 5
  3. 2024Aligning AI Research with the Needs of Clinical Coding Workflows: Eight Recommendations Based on US Data Analysis and Critical Reviewcited by 5
  4. 2023Towards Semi-Structured Automatic ICD Coding via Tree-based Contrastive Learningcited by 4
  5. 2025Toward Reliable Clinical Coding with Language Models: Verification and Lightweight Adaptationcited by 4
  6. 2024Infusing clinical knowledge into tokenisers for language modelscited by 2
  7. 2024Similarity-Dissimilarity Loss for Multi-label Supervised Contrastive Learningcited by 2
  8. 2025TACL: Threshold-Adaptive Curriculum Learning Strategy for Enhancing Medical Text Understandingcited by 2
  9. 2023Automated Clinical Coding for Outpatient Departmentscited by 1
  10. 2024Bespoke Large Language Models for Digital Triage Assistance in Mental Health Carecited by 1
  11. 2024NoteContrast: Contrastive Language-Diagnostic Pretraining for Medical Textcited by 1
  12. 2025Enhancing Biomedical Relation Extraction with Directionalitycited by 1
  13. 2025RuCCoD: Towards Automated ICD Coding in Russiancited by 1
  14. 2026Evaluating ChatGPT on Medical Information Extraction Tasks: Performance, Explainability and Beyondcited by 1
  15. 2026A medical coding language model trained on clinical narratives from a population-wide cohort of 1.8 million patientscited by 1
  16. 2026Retrieve, Then Classify: Corpus-Grounded Automation of Clinical Value Set Authoringcited by 1
  17. 2023Automatic ICD-10 Code Association: A Challenging Task on French Clinical Textscited by 0
  18. 2023ComplicaCode: Enhancing Disease Complication Detection in Electronic Health Records through ICD Path Generationcited by 0
  19. 2024Coding historical causes of death data with Large Language Modelscited by 0
  20. 2024Don't Pay Attention, PLANT It: Pretraining Attention via Learning-to-Rankcited by 0
  21. 2024Your Next State-of-the-Art Could Come from Another Domain: A Cross-Domain Analysis of Hierarchical Text Classificationcited by 0
  22. 2025ASMR: Augmenting Life Scenario using Large Generative Models for Robotic Action Reflectioncited by 0
  23. 2025Federated Learning for ICD Classification with Lightweight Models and Pretrained Embeddingscited by 0
  24. 2025Predictive Multimodal Modeling of Diagnoses and Treatments in EHRcited by 0
  25. 2025LTR-ICD: A Ranking-Aware Framework for Automatic ICD Codingcited by 0
  26. 2025TraceCoder: Towards Traceable ICD Coding via Multi-Source Knowledge Integrationcited by 0
  27. 2025Enhancing Rare Codes via Probability-Biased Directed Graph Attention for Long-Tail ICD Codingcited by 0
  28. 2026From Documents to Spans: Scalable Supervision for Evidence-Based ICD Coding with LLMscited by 0
  29. 2026Symphony for Medical Coding: A Next-Generation Agentic System for Scalable and Explainable Medical Codingcited by 0
  30. 2026ShifaMind: A Multiplicative Concept Bottleneck for Interpretable ICD-10 Codingcited by 0
  31. 2026Systematic Evaluation of the Quality of Synthetic Clinical Notes Rephrased by LLMs at Million-Note Scalecited by 0
  32. 2026Automated ICD Classification of Psychiatric Diagnoses: From Classical NLP to Large Language Modelscited by 0
  33. 2026Generalistic or Specific Embeddings, Which is Better? An Empirical Study on Search for Clinical Coding in Non-English Languagescited by 0
  34. 2026A General Multimodal Probing Framework for Data Fusion and Prediction with Frozen Large Language Models: Evidence from Multimodal EHR Datacited by 0
  35. 2026Graph-Constrained Policy Learning for Extreme Clinical Code Predictioncited by 0
  36. 2026LearnActCoder: Role-Aware Error Memory for Adaptive Clinical Coding Agentscited by 0
  37. 2026Decomposing Error and Style in Automated Clinical Codingcited by 0