mlsanityData to 14 Oct 2026

NeuroMosaic: Anatomically Grounded Multimodal Large Language Modeling for Molecularly Aware Glioma Reasoning from 3D MRI and Clinical Narratives

arXiv 2608.03187 ↗First posted 4 Aug 2026cs.NEcited by 0

On the map: LLM reasoning → Clinical and biomedical applications

Abstract (arXiv)

Multimodal medical large language models remain structurally weak for neuro-oncology because volumetric evidence is compressed into generic visual tokens and diagnostic conclusions often lack an auditable link to MRI regions. We present NeuroMosaic, a 3D multimodal language model that converts multi-sequence brain MRI into anatomy-indexed regional tokens, aligns them with clinical narrative and molecular concepts, and generates evidence-linked outputs. The architecture combines a multi-resolution volumetric tokenizer, a neuroanatomical graph router, a molecular concept memory, and selective risk control. Across four glioma cohorts, NeuroMosaic achieved an internal subtype macro-F1 of 0.827 and external macro-F1 values of 0.784, 0.761, and 0.742. On UPenn-GBM, it improved over the strongest matched-input baseline by 3.6 percentage points (95% CI: 1.8 to 5.4, adjusted p = 0.0018), with IDH, 1p/19q, and MGMT AUROCs of 0.918, 0.861, and 0.781. Evidence pointing accuracy reached 0.703, and targeted evidence deletion reduced correct-answer probability by 0.187, compared with 0.046 for random deletion. These results establish anatomy-indexed routing as a measurable mechanism for accurate, grounded, and calibrated volumetric medical-language reasoning.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Clinical and biomedical applications, one of 17 directions in LLM reasoning. Builds on 19 of its 34 references that are in the map. First posted on arXiv 4 Aug 2026.

Builds on

19 of 34 references.

References · 19

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  3. 2023Visual Instruction Tuningcited by 7,869
  4. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  5. 2017On Calibration of Modern Neural Networkscited by 4,960
  6. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  7. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Daycited by 1,305
  8. 2022Swin UNETR: Swin Transformers for Semantic Segmentation of Brain Tumors in MRI Imagescited by 830
  9. 2020Contrastive Learning of Medical Visual Representations from Paired Images and Textcited by 641
  10. 2019MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographscited by 597
  11. 2022MedCLIP: Contrastive Learning from Unpaired Medical Images and Textcited by 570
  12. 2023BiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairscited by 520
7 more
  1. 2023Med-Flamingo: a Multimodal Medical Few-shot Learnercited by 392
  2. 2020Generating Radiology Reports via Memory-driven Transformercited by 368
  3. 2022Making the Most of Text Semantics to Improve Biomedical Vision--Language Processingcited by 339
  4. 2020CheXbert: Combining Automatic Labelers and Expert Annotations for Accurate Radiology Report Labeling Using BERTcited by 337
  5. 2021RadGraph: Extracting Clinical Entities and Relations from Radiology Reportscited by 319
  6. 2023Towards Generalist Biomedical AIcited by 289
  7. 2021TransBTS: Multimodal Brain Tumor Segmentation Using Transformercited by 265

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026NeuroVLM-Bench: Evaluation of Vision-Enabled Large Language Models for Clinical Reasoning in Neurological Disorders
  2. 2026Evidence-Bound Reasoning: Neuro-Semantic Verification of Biomedical AI in Glioblastoma Radiogenomics
  3. 2026Brain3D: Brain Report Automation via Inflated Vision Transformers in 3D
  4. 2026Text-Guided Refinement of Multi-sequence Glioma Subregion Segmentation with a Vision-Language Foundation Model
  5. 2026NeuroQA: A Large-Scale Image-Grounded Benchmark for 3D Brain MRI Understanding
  6. 2026Agentic Large Language Models for Training-Free Neuro-Radiological Image Analysis
  7. 2024Medical Vision-Language Pre-Training for Brain Abnormalities
  8. 2026Visual Instruction-Finetuned Language Model for Versatile Brain MR Image Tasks