mlsanityData to 14 Oct 2026

DL3M: A Vision-to-Language Framework for Expert-Level Medical Reasoning through Deep Learning and Large Language Models

arXiv 2512.13742 ↗First posted 14 Dec 2025cs.CV cs.AIcited by 0

On the map: LLM reasoning → Clinical and biomedical applications

Abstract (arXiv)

Medical image classifiers detect gastrointestinal diseases well, but they do not explain their decisions. Large language models can generate clinical text, yet they struggle with visual reasoning and often produce unstable or incorrect explanations. This leaves a gap between what a model sees and the type of reasoning a clinician expects. We introduce a framework that links image classification with structured clinical reasoning. A new hybrid model, MobileCoAtNet, is designed for endoscopic images and achieves high accuracy across eight stomach-related classes. Its outputs are then used to drive reasoning by several LLMs. To judge this reasoning, we build two expert-verified benchmarks covering causes, symptoms, treatment, lifestyle, and follow-up care. Thirty-two LLMs are evaluated against these gold standards. Strong classification improves the quality of their explanations, but none of the models reach human-level stability. Even the best LLMs change their reasoning when prompts vary. Our study shows that combining DL with LLMs can produce useful clinical narratives, but current LLMs remain unreliable for high-stakes medical decisions. The framework provides a clearer view of their limits and a path for building safer reasoning systems. The complete source code and datasets used in this study are available at https://github.com/souravbasakshuvo/DL3M.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Clinical and biomedical applications, one of 17 directions in LLM reasoning. Builds on 22 of its 67 references that are in the map. First posted on arXiv 14 Dec 2025.

Builds on

22 of 67 references.

References · 22

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  4. 2025Qwen3 Technical Reportcited by 8,820
  5. 2016Densely Connected Convolutional Networkscited by 6,720
  6. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  7. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  8. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,475
  9. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
  10. 2016Xception: Deep Learning with Depthwise Separable Convolutionscited by 2,332
  11. 2024Qwen2 Technical Reportcited by 2,120
  12. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
10 more
  1. 2019Searching for MobileNetV3cited by 1,994
  2. 2016Identity Mappings in Deep Residual Networkscited by 1,965
  3. 2016Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learningcited by 1,512
  4. 2021EfficientNetV2: Smaller Models and Faster Trainingcited by 1,135
  5. 2023TELeR: A General Taxonomy of LLM Prompts for Benchmarking Complex Taskscited by 49
  6. 2024A review of deep learning-based information fusion techniques for multimodal medical image classificationcited by 25
  7. 2025Pitfalls of Evaluating Language Models with Open Benchmarkscited by 7
  8. 2024Investigating Annotator Bias in Large Language Models for Hate Speech Detectioncited by 6
  9. 2025Zero-Shot Multi-Label Classification of Bangla Documents: Large Decoders Vs. Classic Encoderscited by 2
  10. 2025Investigating Hallucination in Conversations for Low Resource Languagescited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Analogical Reasoning as a Doctor: A Foundation Model for Gastrointestinal Endoscopy Diagnosis
  2. 2025DiagR1: A Vision-Language Model Trained via Reinforcement Learning for Digestive Pathology Diagnosis
  3. 2025Kvasir-VQA-x1: A Multimodal Dataset for Medical Reasoning and Robust MedVQA in Gastrointestinal Endoscopy
  4. 2024Vision-Language and Large Language Model Performance in Gastroenterology: GPT, Claude, Llama, Phi, Mistral, Gemma, and Quantized Models
  5. 2026Parameter-Efficient VLMs for Gastrointestinal Endoscopy: Medical Image Generation and Clinical Visual Question Answering
  6. 2026A visual large language foundational model for medical image recognition using clinician-contributed online resources
  7. 2026ARISE: Adaptive Agentic Reasoning with Image-grounded Self-Evaluation for Interpretable IBD Assessment
  8. 2026Clinical Cognition Alignment for Gastrointestinal Diagnosis with Multimodal LLMs