mlsanityData to 7 Oct 2026

Explaining Chest X-ray Pathologies in Natural Language

arXiv 2207.04343 ↗First posted 9 Jul 2022cs.CV cs.AI cs.CLcited by 27

Not on the map. Why?

Abstract (arXiv)

Most deep learning algorithms lack explanations for their predictions, which limits their deployment in clinical practice. Approaches to improve explainability, especially in medical imaging, have often been shown to convey limited information, be overly reassuring, or lack robustness. In this work, we introduce the task of generating natural language explanations (NLEs) to justify predictions made on medical images. NLEs are human-friendly and comprehensive, and enable the training of intrinsically explainable models. To this goal, we introduce MIMIC-NLE, the first, large-scale, medical imaging dataset with NLEs. It contains over 38,000 NLEs, which explain the presence of various thoracic pathologies and chest X-ray findings. We propose a general approach to solve the task and evaluate several architectures on this dataset, including via clinician assessment.

Key facts. Cited by 27 papers in the map (data through 7 Oct 2026). Builds on 26 of its 40 references that are in the map. First posted on arXiv 9 Jul 2022.

Builds on

26 of 40 references.

References · 26

  1. 2016Densely Connected Convolutional Networkscited by 6,691
  2. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  3. 2019CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparisoncited by 1,410
  4. 2017Explanation in Artificial Intelligence: Insights from the Social Sciencescited by 1,201
  5. 2018Sanity Checks for Saliency Mapscited by 1,149
  6. 2020Concept Bottleneck Modelscited by 1,022
  7. 2019MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographscited by 590
  8. 2018e-SNLI: Natural Language Inference with Natural Language Explanationscited by 468
  9. 2016Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samplescited by 439
  10. 2019Explain Yourself! Leveraging Language Models for Commonsense Reasoningcited by 367
  11. 2020CheXbert: Combining Automatic Labelers and Expert Annotations for Accurate Radiology Report Labeling Using BERTcited by 329
  12. 2018Textual Explanations for Self-Driving Vehiclescited by 226
14 more
  1. 2020Explainable Automated Fact-Checking for Public Health Claimscited by 209
  2. 2018Multimodal Explanations: Justifying Decisions and Pointing to the Evidencecited by 179
  3. 2018TieNet: Text-Image Embedding Network for Common Thorax Disease Classification and Reporting in Chest X-rayscited by 139
  4. 2021TorchXRayVision: A library of chest X-ray datasets and modelscited by 131
  5. 2021e-ViL: A Dataset and Benchmark for Natural Language Explanations in Vision-Language Taskscited by 84
  6. 2021XProtoNet: Diagnosis in Chest Radiography with Global and Local Explanationscited by 72
  7. 2020Natural Language Rationales with Full-Stack Visual Reasoning: From Pixels to Semantic Frames to Commonsense Graphscited by 46
  8. 2021Using StyleGAN for Visual Interpretability of Deep Learning Models on Medical Imagescited by 35
  9. 2021RATCHET: Medical Transformer for Chest X-ray Diagnosis and Reportingcited by 34
  10. 2021Knowledge-Grounded Self-Rationalization via Extractive and Natural Language Explanationscited by 24
  11. 2019Generation of Multimodal Justification Using Visual Word Constraint Model for Explainable Computer-Aided Diagnosiscited by 18
  12. 2020Interpretable Deep Models for Cardiac Resynchronisation Therapy Response Predictioncited by 17
  13. 2019Saliency Methods for Explaining Adversarial Attackscited by 16
  14. 2018Producing radiologist-quality reports for interpretable artificial intelligencecited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 9; 2024: 12; 2025: 3; 2026: 3.

27 papers.

Sort:

By area:

  1. 2024Vision-Language Models for Medical Report Generation and Visual Question Answering: A Reviewcited by 121
  2. 2023Faithfulness Tests for Natural Language Explanationscited by 107
  3. 2023Rigorously Assessing Natural Language Explanations of Neuronscited by 44
  4. 2024The Probabilities Also Matter: A More Faithful Metric for Faithfulness of Free-Text Explanations in Large Language Modelscited by 29
  5. 2023Xplainer: From X-Ray Observations to Explainable Zero-Shot Diagnosiscited by 23
  6. 2023The Impact of Imperfect XAI on Human-AI Decision-Makingcited by 21
  7. 2024Multimodal Large Language Model driven Radiology Report Generation with Clinical Knowledge Enhancementcited by 16
  8. 2025Multimodal Large Language Models for Medicine: A Comprehensive Surveycited by 16
  9. 2023Vision-Language Modelling For Radiological Imaging and Reports In The Low Data Regimecited by 15
  10. 2024LLaVA Needs More Knowledge: Retrieval Augmented Natural Language Generation with Knowledge Graph for Explaining Thoracic Pathologiescited by 11
  11. 2024CT-AGRG: Automated Abnormality-Guided Report Generation from 3D Chest CT Volumescited by 10
  12. 2024Faithful and Plausible Natural Language Explanations for Image Classification: A Pipeline Approachcited by 7
15 more
  1. 2024Fool Me Once? Contrasting Textual and Visual Explanations in a Clinical Decision-Support Settingcited by 5
  2. 2023KNOW How to Make Up Your Mind! Adversarially Detecting and Alleviating Inconsistencies in Natural Language Explanationscited by 4
  3. 2026A Reasoning-Enabled Vision-Language Foundation Model for Chest X-ray Interpretationcited by 4
  4. 2024TiBiX: Leveraging Temporal Information for Bidirectional X-ray and Report Generationcited by 3
  5. 2024Learning To Guide Human Decision Makers With Vision-Language Modelscited by 3
  6. 2023SPARSEFIT: Few-shot Prompting with Sparse Fine-tuning for Jointly Generating Predictions and Natural Language Explanationscited by 2
  7. 2023Using Natural Language Explanations to Improve Robustness of In-context Learningcited by 2
  8. 2024Towards Efficient and Robust VQA-NLE Data Generation with Large Vision-Language Modelscited by 2
  9. 2024EchoNarrator: Generating natural text explanations for ejection fraction predictionscited by 2
  10. 2025Explainable Artificial Intelligence in Biomedical Image Analysis: A Comprehensive Surveycited by 1
  11. 2023RaDialog: A Large Vision-Language Model for Radiology Report Generation and Conversational Assistancecited by 0
  12. 2024Explaining Chest X-ray Pathology Models using Textual Conceptscited by 0
  13. 2025MedVision: Benchmarking Quantitative Medical Image Analysiscited by 0
  14. 2026WeNLEX: Weakly Supervised Natural Language Explanations for Multilabel Chest X-ray Classificationcited by 0
  15. 2026Consistent but Miscalibrated: Evaluating LLM Limitations for Risk Communication in Natural Languagecited by 0