mlsanityData to 7 Oct 2026

BERT-XML: Large Scale Automated ICD Coding Using BERT Pretraining

arXiv 2006.03685 ↗First posted 26 May 2020cs.IR cs.CL cs.LGcited by 37

Not on the map. Why?

Abstract (arXiv)

Clinical interactions are initially recorded and documented in free text medical notes. ICD coding is the task of classifying and coding all diagnoses, symptoms and procedures associated with a patient's visit. The process is often manual and extremely time-consuming and expensive for hospitals. In this paper, we propose a machine learning model, BERT-XML, for large scale automated ICD coding from EHR notes, utilizing recently developed unsupervised pretraining that have achieved state of the art performance on a variety of NLP tasks. We train a BERT model from scratch on EHR notes, learning with vocabulary better suited for EHR tasks and thus outperform off-the-shelf models. We adapt the BERT architecture for ICD coding with multi-label attention. While other works focus on small public medical datasets, we have produced the first large scale ICD-10 classification model using millions of EHR notes to predict thousands of unique ICD codes.

Key facts. Cited by 37 papers in the map (data through 7 Oct 2026). First posted on arXiv 26 May 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 16; 2023: 7; 2024: 4; 2025: 4; 2026: 2.

37 papers.

Sort:

  1. 2022R3M: A Universal Visual Representation for Robot Manipulationcited by 678
  2. 2022Large Language Models are Few-Shot Clinical Information Extractorscited by 206
  3. 2022Clinical-Longformer and Clinical-BigBird: Transformers for long clinical sequencescited by 67
  4. 2022PLM-ICD: Automatic ICD Coding with Pretrained Language Modelscited by 49
  5. 2023Automated Medical Coding on MIMIC-III and MIMIC-IV: A Critical Review and Replicability Studycited by 48
  6. 2021Clinical Outcome Prediction from Admission Notes using Self-Supervised Knowledge Integrationcited by 43
  7. 2022Knowledge Injected Prompt Based Fine-tuning for Multi-label Few-shot ICD Codingcited by 40
  8. 2021Neural Natural Language Processing for Unstructured Data in Electronic Health Records: a Reviewcited by 35
  9. 2023A Comparative Study of Pretrained Language Models for Long Clinical Textcited by 35
  10. 2022A Unified Review of Deep Learning for Automated Medical Codingcited by 26
  11. 2024Large Language Models in Drug Discovery and Development: From Disease Mechanisms to Clinical Trialscited by 24
  12. 2021Read, Attend, and Code: Pushing the Limits of Medical Codes Prediction from Clinical Notes by Machinescited by 23
25 more
  1. 2023Transformers in Healthcare: A Surveycited by 17
  2. 2022Multi-label Few-shot ICD Coding as Autoregressive Generation with Promptcited by 16
  3. 2022Multimodal data matters: language model pre-training over structured and unstructured electronic health recordscited by 13
  4. 2022Hierarchical Label-wise Attention Transformer Model for Explainable ICD Codingcited by 13
  5. 2023A Review on Knowledge Graphs for Healthcare: Resources, Applications, and Promisescited by 13
  6. 2021A Systematic Literature Review of Automated ICD Coding and Classification Systems using Discharge Summariescited by 10
  7. 2022ICDBigBird: A Contextual Embedding Model for ICD Code Classificationcited by 10
  8. 2022Automated ICD Coding using Extreme Multi-label Long Text Transformer-based Modelscited by 5
  9. 2023TreeMAN: Tree-enhanced Multimodal Attention Network for ICD Codingcited by 5
  10. 2024GAMedX: Generative AI-based Medical Entity Data Extractor Using Large Language Modelscited by 5
  11. 2024Aligning AI Research with the Needs of Clinical Coding Workflows: Eight Recommendations Based on US Data Analysis and Critical Reviewcited by 5
  12. 2022BERT for Long Documents: A Case Study of Automated ICD Codingcited by 4
  13. 2022Entity Anchored ICD Codingcited by 3
  14. 2024Contrastive Learning on Multimodal Analysis of Electronic Health Recordscited by 2
  15. 2022A Scalable Workflow to Build Machine Learning Classifiers with Clinician-in-the-Loop to Identify Patients in Specific Diseasescited by 1
  16. 2023Automated Clinical Coding for Outpatient Departmentscited by 1
  17. 2025RuCCoD: Towards Automated ICD Coding in Russiancited by 1
  18. 2025CURENet: Combining Unified Representations for Efficient Chronic Disease Predictioncited by 1
  19. 2022Semi-self-supervised Automated ICD Codingcited by 0
  20. 2022GrabQC: Graph based Query Contextualization for automated ICD codingcited by 0
  21. 2023Multimodal Recommender Systems in the Prediction of Disease Comorbiditycited by 0
  22. 2025MKE-Coder: Multi-Axial Knowledge with Evidence Verification in ICD Coding for Chinese EMRscited by 0
  23. 2025LTR-ICD: A Ranking-Aware Framework for Automatic ICD Codingcited by 0
  24. 2026Can Post-Training Turn LLMs into Good Medical Coders? An Empirical Study of Generative ICD Codingcited by 0
  25. 2026CoLa-ICD: A Knowledge-Enhanced Framework for Long-Tail Automated Medical Codingcited by 0