# Multimodal Machine Learning for Automated ICD Coding

arXiv: https://arxiv.org/abs/1810.13348 · HTML: https://mlsanity.com/paper/1810.13348/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Oct 2018.

## Built on by (most cited first, 30 of 41)

- [MultiBench: Multiscale Benchmarks for Multimodal Representation Learning](https://mlsanity.com/paper/2107.07502/) (2021)
- [MultiViz: Towards Visualizing and Understanding Multimodal Models](https://mlsanity.com/paper/2207.00056/) (2022)
- [Neural Natural Language Processing for Unstructured Data in Electronic Health Records: a Review](https://mlsanity.com/paper/2107.02975/) (2021)
- [DIME: Fine-grained Interpretations of Multimodal Models via Disentangled Local Explanations](https://mlsanity.com/paper/2203.02013/) (2022)
- [Automated Clinical Coding: What, Why, and Where We Are?](https://mlsanity.com/paper/2203.11092/) (2022)
- [A Unified Review of Deep Learning for Automated Medical Coding](https://mlsanity.com/paper/2201.02797/) (2022)
- [Imbalanced Big Data Oversampling: Taxonomy, Algorithms, Software, Guidelines and Future Directions](https://mlsanity.com/paper/2107.11508/) (2021)
- [HEMM: Holistic Evaluation of Multimodal Foundation Models](https://mlsanity.com/paper/2407.03418/) (2024)
- [An Unsupervised Approach to Achieve Supervised-Level Explainability in Healthcare Records](https://mlsanity.com/paper/2406.08958/) (2024)
- [A Systematic Literature Review of Automated ICD Coding and Classification Systems using Discharge Summaries](https://mlsanity.com/paper/2107.10652/) (2021)
- [Large language models are good medical coders, if provided with tools](https://mlsanity.com/paper/2407.12849/) (2024)
- [Attack-agnostic Adversarial Detection on Medical Data Using Explainable Machine Learning](https://mlsanity.com/paper/2105.01959/) (2021)
- [Machine Learning for Multimodal Electronic Health Records-based Research: Challenges and Perspectives](https://mlsanity.com/paper/2111.04898/) (2021)
- [Classifying Unstructured Clinical Notes via Automatic Weak Supervision](https://mlsanity.com/paper/2206.12088/) (2022)
- [MuG: A Multimodal Classification Benchmark on Game Data with Tabular, Textual, and Visual Fields](https://mlsanity.com/paper/2302.02978/) (2023)
- [TreeMAN: Tree-enhanced Multimodal Attention Network for ICD Coding](https://mlsanity.com/paper/2305.18576/) (2023)
- [SimSUM: Simulated Benchmark with Structured and Unstructured Medical Records](https://mlsanity.com/paper/2409.08936/) (2024)
- [MultiZoo & MultiBench: A Standardized Toolkit for Multimodal Deep Learning](https://mlsanity.com/paper/2306.16413/) (2023)
- [Brainish: Formalizing A Multimodal Language for Intelligence and Consciousness](https://mlsanity.com/paper/2205.00001/) (2022)
- [Examining Imbalance Effects on Performance and Demographic Fairness of Clinical Language Models](https://mlsanity.com/paper/2412.17803/) (2024)
- [Multimodal Feature Fusion for Video Advertisements Tagging Via Stacking Ensemble](https://mlsanity.com/paper/2108.00679/) (2021)
- [Multimodal Machine Learning in Precision Health](https://mlsanity.com/paper/2204.04777/) (2022)
- [HieNet: Bidirectional Hierarchy Framework for Automated ICD Coding](https://mlsanity.com/paper/2212.04891/) (2022)
- [A Meta-embedding-based Ensemble Approach for ICD Coding Prediction](https://mlsanity.com/paper/2102.13622/) (2021)
- [Large Language Model in Medical Informatics: Direct Classification and Enhanced Text Representations for Automatic ICD Coding](https://mlsanity.com/paper/2411.06823/) (2024)
- [Few-Shot Electronic Health Record Coding through Graph Contrastive Learning](https://mlsanity.com/paper/2106.15467/) (2021)
- [A Scalable Workflow to Build Machine Learning Classifiers with Clinician-in-the-Loop to Identify Patients in Specific Diseases](https://mlsanity.com/paper/2205.08891/) (2022)
- [Automated Fidelity Assessment for Strategy Training in Inpatient Rehabilitation using Natural Language Processing](https://mlsanity.com/paper/2209.06727/) (2022)
- [RuCCoD: Towards Automated ICD Coding in Russian](https://mlsanity.com/paper/2502.21263/) (2025)
- [From Extreme Multi-label to Multi-class: A Hierarchical Approach for Automated ICD-10 Coding Using Phrase-level Attention](https://mlsanity.com/paper/2102.09136/) (2021)

## Abstract (arXiv)

This study presents a multimodal machine learning model to predict ICD-10 diagnostic codes. We developed separate machine learning models that can handle data from different modalities, including unstructured text, semi-structured text and structured tabular data. We further employed an ensemble method to integrate all modality-specific models to generate ICD-10 codes. Key evidence was also extracted to make our prediction more convincing and explainable. We used the Medical Information Mart for Intensive Care III (MIMIC -III) dataset to validate our approach. For ICD code prediction, our best-performing model (micro-F1 = 0.7633, micro-AUC = 0.9541) significantly outperforms other baseline models including TF-IDF (micro-F1 = 0.6721, micro-AUC = 0.7879) and Text-CNN model (micro-F1 = 0.6569, micro-AUC = 0.9235). For interpretability, our approach achieves a Jaccard Similarity Coefficient (JSC) of 0.1806 on text data and 0.3105 on tabular data, where well-trained physicians achieve 0.2780 and 0.5002 respectively.