Experimental Evaluation and Development of a Silver-Standard for the MIMIC-III Clinical Coding Dataset
Not on the map. Why?
Abstract (arXiv)
Clinical coding is currently a labour-intensive, error-prone, but critical administrative process whereby hospital patient episodes are manually assigned codes by qualified staff from large, standardised taxonomic hierarchies of codes. Automating clinical coding has a long history in NLP research and has recently seen novel developments setting new state of the art results. A popular dataset used in this task is MIMIC-III, a large intensive care database that includes clinical free text notes and associated codes. We argue for the reconsideration of the validity MIMIC-III's assigned codes that are often treated as gold-standard, especially when MIMIC-III has not undergone secondary validation. This work presents an open-source, reproducible experimental methodology for assessing the validity of codes derived from EHR discharge summaries. We exemplify the methodology with MIMIC-III discharge summaries and show the most frequently assigned codes in MIMIC-III are under-coded up to 35%.
Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). First posted on arXiv 12 Jun 2020.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 3; 2022: 5; 2023: 3; 2024: 2; 2025: 1; 2026: 1.
15 papers.
Sort:
By area:
- 2023Automated Medical Coding on MIMIC-III and MIMIC-IV: A Critical Review and Replicability Studycited by 48
- 2021Clinical Outcome Prediction from Admission Notes using Self-Supervised Knowledge Integrationcited by 43
- 2022Automated Clinical Coding: What, Why, and Where We Are?cited by 33
- 2022A Unified Review of Deep Learning for Automated Medical Codingcited by 26
- 2023MDACE: MIMIC Documents Annotated with Code Evidencecited by 19
- 2022Discharge Summary Hospital Course Summarisation of In Patient Electronic Health Record Text with Clinical Concept Guided Deep Pre-Trained Transformer Modelscited by 17
- 2021A Systematic Literature Review of Automated ICD Coding and Classification Systems using Discharge Summariescited by 10
- 2025Code Like Humans: A Multi-Agent Solution for Medical Codingcited by 10
- 2022Ontology-Driven and Weakly Supervised Rare Disease Identification from Clinical Notescited by 7
- 2021Rare Disease Identification from Clinical Notes with Ontologies and Weak Supervisioncited by 5
- 2022Automated ICD Coding using Extreme Multi-label Long Text Transformer-based Modelscited by 5
- 2024Aligning AI Research with the Needs of Clinical Coding Workflows: Eight Recommendations Based on US Data Analysis and Critical Reviewcited by 5
- 2026Understanding the Limits of Agentic ICD Coding12 Sep 2026
- 2025Code Like Humans: A Multi-Agent Solution for Medical Coding4 Sep 2025
- 2024Aligning AI Research with the Needs of Clinical Coding Workflows: Eight Recommendations Based on US Data Analysis and Critical Review23 Dec 2024
- 2024Infusing clinical knowledge into tokenisers for language models20 Jun 2024
- 2023Exploring the Consistency, Quality and Challenges in Manual and Automated Coding of Free-text Diagnoses from Hospital Outpatient Letters17 Nov 2023
- 2023MDACE: MIMIC Documents Annotated with Code Evidence7 Jul 2023
- 2023Automated Medical Coding on MIMIC-III and MIMIC-IV: A Critical Review and Replicability Study21 Apr 2023
- 2022Automated ICD Coding using Extreme Multi-label Long Text Transformer-based Models12 Dec 2022
- 2022Discharge Summary Hospital Course Summarisation of In Patient Electronic Health Record Text with Clinical Concept Guided Deep Pre-Trained Transformer Models14 Nov 2022
- 2022Ontology-Driven and Weakly Supervised Rare Disease Identification from Clinical Notes11 May 2022
- 2022Automated Clinical Coding: What, Why, and Where We Are?21 Mar 2022
- 2022A Unified Review of Deep Learning for Automated Medical Coding8 Jan 2022
3 more
- 2021A Systematic Literature Review of Automated ICD Coding and Classification Systems using Discharge Summaries12 Jul 2021
- 2021Rare Disease Identification from Clinical Notes with Ontologies and Weak Supervision5 May 2021
- 2021Clinical Outcome Prediction from Admission Notes using Self-Supervised Knowledge Integration8 Feb 2021
- 2025Code Like Humans: A Multi-Agent Solution for Medical Codingcited by 10
- 2026Understanding the Limits of Agentic ICD Codingcited by 0
- 2026Understanding the Limits of Agentic ICD Coding12 Sep 2026
- 2025Code Like Humans: A Multi-Agent Solution for Medical Coding4 Sep 2025
- 2023Automated Medical Coding on MIMIC-III and MIMIC-IV: A Critical Review and Replicability Studycited by 48
- 2021Clinical Outcome Prediction from Admission Notes using Self-Supervised Knowledge Integrationcited by 43
- 2022Automated Clinical Coding: What, Why, and Where We Are?cited by 33
- 2022A Unified Review of Deep Learning for Automated Medical Codingcited by 26
- 2023MDACE: MIMIC Documents Annotated with Code Evidencecited by 19
- 2022Discharge Summary Hospital Course Summarisation of In Patient Electronic Health Record Text with Clinical Concept Guided Deep Pre-Trained Transformer Modelscited by 17
- 2021A Systematic Literature Review of Automated ICD Coding and Classification Systems using Discharge Summariescited by 10
- 2022Ontology-Driven and Weakly Supervised Rare Disease Identification from Clinical Notescited by 7
- 2021Rare Disease Identification from Clinical Notes with Ontologies and Weak Supervisioncited by 5
- 2022Automated ICD Coding using Extreme Multi-label Long Text Transformer-based Modelscited by 5
- 2024Aligning AI Research with the Needs of Clinical Coding Workflows: Eight Recommendations Based on US Data Analysis and Critical Reviewcited by 5
- 2023Exploring the Consistency, Quality and Challenges in Manual and Automated Coding of Free-text Diagnoses from Hospital Outpatient Letterscited by 2
12 most cited of 13 All 13 (JSON)
- 2024Aligning AI Research with the Needs of Clinical Coding Workflows: Eight Recommendations Based on US Data Analysis and Critical Review23 Dec 2024
- 2024Infusing clinical knowledge into tokenisers for language models20 Jun 2024
- 2023Exploring the Consistency, Quality and Challenges in Manual and Automated Coding of Free-text Diagnoses from Hospital Outpatient Letters17 Nov 2023
- 2023MDACE: MIMIC Documents Annotated with Code Evidence7 Jul 2023
- 2023Automated Medical Coding on MIMIC-III and MIMIC-IV: A Critical Review and Replicability Study21 Apr 2023
- 2022Automated ICD Coding using Extreme Multi-label Long Text Transformer-based Models12 Dec 2022
- 2022Discharge Summary Hospital Course Summarisation of In Patient Electronic Health Record Text with Clinical Concept Guided Deep Pre-Trained Transformer Models14 Nov 2022
- 2022Ontology-Driven and Weakly Supervised Rare Disease Identification from Clinical Notes11 May 2022
- 2022Automated Clinical Coding: What, Why, and Where We Are?21 Mar 2022
- 2022A Unified Review of Deep Learning for Automated Medical Coding8 Jan 2022
- 2021A Systematic Literature Review of Automated ICD Coding and Classification Systems using Discharge Summaries12 Jul 2021
- 2021Rare Disease Identification from Clinical Notes with Ontologies and Weak Supervision5 May 2021
12 newest of 13 All 13 (JSON)