# Experimental Evaluation and Development of a Silver-Standard for the MIMIC-III Clinical Coding Dataset

arXiv: https://arxiv.org/abs/2006.07332 · HTML: https://mlsanity.com/paper/2006.07332/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Jun 2020.

## Built on by (most cited first, 15 of 15)

- [Automated Medical Coding on MIMIC-III and MIMIC-IV: A Critical Review and Replicability Study](https://mlsanity.com/paper/2304.10909/) (2023)
- [Clinical Outcome Prediction from Admission Notes using Self-Supervised Knowledge Integration](https://mlsanity.com/paper/2102.04110/) (2021)
- [Automated Clinical Coding: What, Why, and Where We Are?](https://mlsanity.com/paper/2203.11092/) (2022)
- [A Unified Review of Deep Learning for Automated Medical Coding](https://mlsanity.com/paper/2201.02797/) (2022)
- [MDACE: MIMIC Documents Annotated with Code Evidence](https://mlsanity.com/paper/2307.03859/) (2023)
- [Discharge Summary Hospital Course Summarisation of In Patient Electronic Health Record Text with Clinical Concept Guided Deep Pre-Trained Transformer Models](https://mlsanity.com/paper/2211.07126/) (2022)
- [Code Like Humans: A Multi-Agent Solution for Medical Coding](https://mlsanity.com/paper/2509.05378/) (2025)
- [A Systematic Literature Review of Automated ICD Coding and Classification Systems using Discharge Summaries](https://mlsanity.com/paper/2107.10652/) (2021)
- [Ontology-Driven and Weakly Supervised Rare Disease Identification from Clinical Notes](https://mlsanity.com/paper/2205.05656/) (2022)
- [Rare Disease Identification from Clinical Notes with Ontologies and Weak Supervision](https://mlsanity.com/paper/2105.01995/) (2021)
- [Automated ICD Coding using Extreme Multi-label Long Text Transformer-based Models](https://mlsanity.com/paper/2212.05857/) (2022)
- [Aligning AI Research with the Needs of Clinical Coding Workflows: Eight Recommendations Based on US Data Analysis and Critical Review](https://mlsanity.com/paper/2412.18043/) (2024)
- [Exploring the Consistency, Quality and Challenges in Manual and Automated Coding of Free-text Diagnoses from Hospital Outpatient Letters](https://mlsanity.com/paper/2311.10856/) (2023)
- [Infusing clinical knowledge into tokenisers for language models](https://mlsanity.com/paper/2406.14312/) (2024)
- [Understanding the Limits of Agentic ICD Coding](https://mlsanity.com/paper/2609.13806/) (2026)

## Abstract (arXiv)

Clinical coding is currently a labour-intensive, error-prone, but critical administrative process whereby hospital patient episodes are manually assigned codes by qualified staff from large, standardised taxonomic hierarchies of codes. Automating clinical coding has a long history in NLP research and has recently seen novel developments setting new state of the art results. A popular dataset used in this task is MIMIC-III, a large intensive care database that includes clinical free text notes and associated codes. We argue for the reconsideration of the validity MIMIC-III's assigned codes that are often treated as gold-standard, especially when MIMIC-III has not undergone secondary validation. This work presents an open-source, reproducible experimental methodology for assessing the validity of codes derived from EHR discharge summaries. We exemplify the methodology with MIMIC-III discharge summaries and show the most frequently assigned codes in MIMIC-III are under-coded up to 35%.