mlsanityData to 14 Oct 2026

AOR: Anatomical Ontology-Guided Reasoning for Medical Large Multimodal Model in Chest X-Ray Interpretation

arXiv 2505.02830 ↗First posted 5 May 2025cs.CV cs.CLcited by 8

Not on the map. Why?

Abstract (arXiv)

Chest X-rays (CXRs) are the most frequently performed imaging examinations in clinical settings. Recent advancements in Large Multimodal Models (LMMs) have enabled automated CXR interpretation, enhancing diagnostic accuracy and efficiency. However, despite their strong visual understanding, current Medical LMMs (MLMMs) still face two major challenges: (1) Insufficient region-level understanding and interaction, and (2) Limited accuracy and interpretability due to single-step reasoning. In this paper, we empower MLMMs with anatomy-centric reasoning capabilities to enhance their interactivity and explainability. Specifically, we first propose an Anatomical Ontology-Guided Reasoning (AOR) framework, which centers on cross-modal region-level information to facilitate multi-step reasoning. Next, under the guidance of expert physicians, we develop AOR-Instruction, a large instruction dataset for MLMMs training. Our experiments demonstrate AOR's superior performance in both VQA and report generation tasks.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 31 of its 47 references that are in the map. First posted on arXiv 5 May 2025.

Builds on

31 of 47 references.

References · 31

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  6. 2023Visual Instruction Tuningcited by 7,869
  7. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  8. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  9. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  10. 2019CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparisoncited by 1,423
  11. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Daycited by 1,305
  12. 2023Shikra: Unleashing Multimodal LLM's Referential Dialogue Magiccited by 906
19 more
  1. 2017CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learningcited by 461
  2. 2023Med-Flamingo: a Multimodal Medical Few-shot Learnercited by 392
  3. 2020Generating Radiology Reports via Memory-driven Transformercited by 368
  4. 2021RadGraph: Extracting Clinical Entities and Relations from Radiology Reportscited by 319
  5. 2023GPT4RoI: Instruction Tuning Large Language Model on Region-of-Interestcited by 302
  6. 2024Visual CoT: Advancing Multi-Modal Language Models with a Comprehensive Dataset and Benchmark for Chain-of-Thought Reasoningcited by 280
  7. 2020VinDr-CXR: An open dataset of chest X-rays with radiologist's annotationscited by 239
  8. 2025MedVLM-R1: Incentivizing Medical Reasoning Capability of Vision-Language Models (VLMs) via Reinforcement Learningcited by 173
  9. 2023Med-HALT: Medical Domain Hallucination Test for Large Language Modelscited by 157
  10. 2024MAIRA-2: Grounded Radiology Report Generationcited by 145
  11. 2020Improving Factual Completeness and Consistency of Image-to-Text Radiology Report Generationcited by 141
  12. 2021Chest ImaGenome Dataset for Clinical Reasoningcited by 90
  13. 2024RegionGPT: Towards Region Understanding Vision Language Modelcited by 71
  14. 2023EHRXQA: A Multi-Modal Question Answering Dataset for Electronic Health Records with Chest X-ray Imagescited by 53
  15. 2024MedCoT: Medical Chain of Thought via Hierarchical Expertcited by 46
  16. 2024VoCoT: Unleashing Visually Grounded Multi-Step Reasoning in Large Multi-Modal Modelscited by 43
  17. 2024Anatomical Structure-Guided Medical Vision-Language Pre-trainingcited by 18
  18. 2024Probabilistic Medical Predictions of Large Language Modelscited by 16
  19. 2024A Refer-and-Ground Multimodal Large Language Model for Biomedicinecited by 14

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 5; 2026: 3.

8 papers.

  1. 2025Thinking with Images for Multimodal Reasoning: Foundations, Methods, and Future Frontierscited by 199
  2. 2025OralGPT-Omni: A Versatile Dental Multimodal Large Language Modelcited by 11
  3. 2025Medical Reasoning in the Era of LLMs: A Systematic Review of Enhancement Techniques and Applicationscited by 10
  4. 2025Beyond MedQA: Towards Real-world Clinical Decision Making in the Era of LLMscited by 1
  5. 2026AnatomiX, an Anatomy-Aware Grounded Multimodal Large Language Model for Chest X-Ray Interpretationcited by 1
  6. 2025Adapting and Evaluating Multimodal Large Language Models for Adolescent Idiopathic Scoliosis Self-Management: A Divide and Conquer Frameworkcited by 0
  7. 2026Towards Responsible Multimodal Medical Reasoning via Context-Aligned Vision-Language Modelscited by 0
  8. 2026EasyLens: A Training-Free Plug-and-Play Subtle-Lesion Representation Amplifier for Medical Vision-Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025RadAgents: Multimodal Agentic Reasoning for Chest X-ray Interpretation with Radiologist-like Workflows
  2. 2025CXReasonBench: A Benchmark for Evaluating Structured Diagnostic Reasoning in Chest X-rays
  3. 2026Rethinking Visual Attribution for Chest X-ray Reasoning in Large Vision Language Models
  4. 2026A Reasoning-Enabled Vision-Language Foundation Model for Chest X-ray Interpretation
  5. 2025Interpreting Chest X-rays Like a Radiologist: A Benchmark with Clinical Reasoning
  6. 2025RadFabric: Agentic AI System with Reasoning Capability for Radiology
  7. 2025CX-Mind: A Pioneering Multimodal Large Language Model for Interleaved Reasoning in Chest X-ray via Curriculum-Guided Reinforcement Learning
  8. 2025Reasoning Visual Language Model for Chest X-Ray Analysis