mlsanityData to 14 Oct 2026

Towards Interpretable Radiology Report Generation via Concept Bottlenecks using a Multi-Agentic RAG

arXiv 2412.16086 ↗First posted 20 Dec 2024cs.IR cs.AI cs.CLcited by 9

On the map: Retrieval-augmented generation → Medical and biomedical RAG

Abstract (arXiv)

Deep learning has advanced medical image classification, but interpretability challenges hinder its clinical adoption. This study enhances interpretability in Chest X-ray (CXR) classification by using concept bottleneck models (CBMs) and a multi-agent Retrieval-Augmented Generation (RAG) system for report generation. By modeling relationships between visual features and clinical concepts, we create interpretable concept vectors that guide a multi-agent RAG system to generate radiology reports, enhancing clinical relevance, explainability, and transparency. Evaluation of the generated reports using an LLM-as-a-judge confirmed the interpretability and clinical utility of our model's outputs. On the COVID-QU dataset, our model achieved 81% classification accuracy and demonstrated robust report generation performance, with five key metrics ranging between 84% and 90%. This interpretable multi-agent framework bridges the gap between high-performance AI and the explainability required for reliable AI-driven CXR analysis in clinical settings. Our code is available at https://github.com/tifat58/IRR-with-CBM-RAG.git.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Placed in Medical and biomedical RAG, one of 15 directions in Retrieval-augmented generation. Builds on 26 of its 41 references that are in the map. First posted on arXiv 20 Dec 2024.

Builds on

26 of 41 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024MMed-RAG: Versatile Multimodal RAG System for Medical Vision Language Models16 Oct
  2. 2023Retrieval Augmented Chest X-Ray Report Generation using OpenAI GPT models5 May

Other references · 24

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2023GPT-4 Technical Reportcited by 18,396
  4. 2024The Llama 3 Herd of Modelscited by 14,715
  5. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  6. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  7. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  8. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  9. 2023Mistral 7Bcited by 3,164
  10. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
  11. 2022Large Language Models Encode Clinical Knowledgecited by 1,830
  12. 2020Concept Bottleneck Modelscited by 1,032
12 more
  1. 2024Mixture-of-Agents Enhances Large Language Model Capabilitiescited by 438
  2. 2022Post-hoc Concept Bottleneck Modelscited by 293
  3. 2023Label-Free Concept Bottleneck Modelscited by 285
  4. 2017On the Automatic Generation of Medical Imaging Reportscited by 216
  5. 2023Learning to Exploit Temporal Structure for Biomedical Vision-Language Processingcited by 211
  6. 2020Can AI help in screening Viral and COVID-19 pneumonia?cited by 200
  7. 2024More Agents Is All You Needcited by 180
  8. 2022Interactive Concept Bottleneck Modelscited by 79
  9. 2023A Closer Look at the Intervention Procedure of Concept Bottleneck Modelscited by 64
  10. 2023Robust and Interpretable Medical Image Classifiers via Concept Bottleneck Modelscited by 40
  11. 2023Learning to Intervene on Concept Bottleneckscited by 29
  12. 2024Fact-Aware Multimodal Retrieval Augmentation for Accurate Medical Radiology Report Generationcited by 23

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 8; 2026: 1.

9 papers.

  1. 2025A Survey of LLM-based Deep Search Agents: Paradigm, Optimization, Evaluation, and Challengescited by 62
  2. 2025A Multimodal Multi-Agent Framework for Radiology Report Generationcited by 8
  3. 2025SUB: Benchmarking CBM Generalization via Synthetic Attribute Substitutionscited by 6
  4. 2025Engineering RAG Systems for Real-World Applications: Design, Development, and Evaluationcited by 3
  5. 2025CBM-RAG: Demonstrating Enhanced Interpretability in Radiology Report Generation with Multi-Agent RAG and Concept Bottleneck Modelscited by 2
  6. 2025Towards Automated Situation Awareness: A RAG-Based Framework for Peacebuilding Reportscited by 1
  7. 2025A Multi-Agent System for Complex Reasoning in Radiology Visual Question Answeringcited by 1
  8. 2025InFL-UX: A Toolkit for Web-Based Interactive Federated Learningcited by 0
  9. 2026TRACE: A Concept Bottleneck Model for Longitudinal 3D Glioblastoma Response Assessmentcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Concept-Enhanced Multimodal RAG: Towards Interpretable and Accurate Radiology Report Generation
  2. 2024CXR-Agent: Vision-language models for chest X-ray interpretation with uncertainty aware radiology reporting
  3. 2023I-AI: A Controllable & Interpretable AI System for Decoding Radiologists' Intense Focus for Accurate CXR Diagnoses
  4. 2026A unified multi-task framework enables interpretable chest radiograph analysis
  5. 2026CogRad: A Cognitively-Inspired Multi-Agent Framework for Radiology Report Generation
  6. 2026MAC-RRG: Iterative Multi-Agent Collaboration for X-ray Radiology Report Generation
  7. 2025Interpreting Chest X-rays Like a Radiologist: A Benchmark with Clinical Reasoning
  8. 2024M4CXR: Exploring Multi-task Potentials of Multi-modal Large Language Models for Chest X-ray Interpretation