DiA-gnostic VLVAE: Disentangled Alignment-Constrained Vision Language Variational AutoEncoder for Robust Radiology Reporting with Missing Modalities
On the map: Mixture-of-experts → Medical and biomedical MoE
Abstract (arXiv)
The integration of medical images with clinical context is essential for generating accurate and clinically interpretable radiology reports. However, current automated methods often rely on resource-heavy Large Language Models (LLMs) or static knowledge graphs and struggle with two fundamental challenges in real-world clinical data: (1) missing modalities, such as incomplete clinical context , and (2) feature entanglement, where mixed modality-specific and shared information leads to suboptimal fusion and clinically unfaithful hallucinated findings. To address these challenges, we propose the DiA-gnostic VLVAE, which achieves robust radiology reporting through Disentangled Alignment. Our framework is designed to be resilient to missing modalities by disentangling shared and modality-specific features using a Mixture-of-Experts (MoE) based Vision-Language Variational Autoencoder (VLVAE). A constrained optimization objective enforces orthogonality and alignment between these latent representations to prevent suboptimal fusion. A compact LLaMA-X decoder then uses these disentangled representations to generate reports efficiently. On the IU X-Ray and MIMIC-CXR datasets, DiA has achieved competetive BLEU@4 scores of 0.266 and 0.134, respectively. Experimental results show that the proposed method significantly outperforms state-of-the-art models.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Medical and biomedical MoE, one of 12 directions in Mixture-of-experts. Builds on 31 of its 53 references that are in the map. First posted on arXiv 8 Nov 2025.
Builds on
31 of 53 references.
References · 31
- 2017Attention Is All You Needcited by 59,932
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
- 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
- 2019MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographscited by 590
- 2020Generating Radiology Reports via Memory-driven Transformercited by 363
- 2020CheXbert: Combining Automatic Labelers and Expert Annotations for Accurate Radiology Report Labeling Using BERTcited by 329
- 2016Domain Separation Networkscited by 317
- 2021SMIL: Multimodal Learning with Severely Missing Modalitycited by 243
- 2018Multimodal Generative Models for Scalable Weakly-Supervised Learningcited by 200
- 2019Variational Mixture-of-Experts Autoencoders for Multi-Modal Deep Generative Modelscited by 177
19 more
- 2024Exploring scalable medical image encoders beyond text supervisioncited by 129
- 2023Dynamic Graph Enhanced Contrastive Learning for Chest X-ray Report Generationcited by 114
- 2019Missing MRI Pulse Sequence Synthesis using Multi-Modal Generative Adversarial Networkcited by 98
- 2023METransformer: Radiology Report Generation by Transformer with Multiple Learnable Expert Tokenscited by 98
- 2022Improving Chest X-Ray Report Generation by Leveraging Warm Startingcited by 93
- 2023PromptMRG: Diagnosis-Driven Prompts for Medical Report Generationcited by 92
- 2021Radiology Report Generation with a Learned Knowledge Base and Multi-modal Alignmentcited by 71
- 2022M3Care: Learning with Missing Modalities in Multimodal Healthcare Datacited by 71
- 2021Knowledge Matters: Radiology Report Generation with General and Specific Knowledgecited by 68
- 2020Robust Multimodal Brain Tumor Segmentation via Feature Disentanglement and Gated Fusioncited by 67
- 2019Learning Disentangled Representations via Mutual Information Estimationcited by 52
- 2020Multimodal Generative Learning Utilizing Jensen-Shannon-Divergencecited by 48
- 2019Brain Tumor Segmentation on MRI with Missing Modalitiescited by 42
- 2023Multimodal Variational Auto-encoder based Audio-Visual Segmentationcited by 37
- 2021Deep Orthogonal Fusion: Multimodal Prognostic Biomarker Discovery Integrating Radiology, Pathology, Genomic, and Clinical Datacited by 29
- 2022Multi-Modality Cardiac Image Computing: A Surveycited by 19
- 2022Artificial Intelligence-Based Methods for Fusion of Electronic Health Records and Imaging Datacited by 18
- 2024Structural Entities Extraction and Patient Indications Incorporation for Chest X-ray Report Generationcited by 14
- 2025Incomplete Modality Disentangled Representation for Ophthalmic Disease Grading and Diagnosiscited by 6
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026GLINT: Sparsely Gated Vision-Language Alignment for Fine-Grained Radiology Representations
- 2024An X-Ray Is Worth 15 Features: Sparse Autoencoders for Interpretable Radiology Report Generation
- 2025MicarVLMoE: A Modern Gated Cross-Aligned Vision-Language Mixture of Experts Model for Medical Image Captioning and Report Generation
- 2026AlphaRAD: Grounded Zero-Shot Classification in Chest Radiology via α-Corrected Binary Cross Entropy and Factorized Latent Supervision
- 2025Visual Alignment of Medical Vision-Language Models for Grounded Radiology Report Generation
- 2025Exploring the Capabilities of Large Language Model Encoders for Image-Text Retrieval in Chest X-rays
- 2026Universal Boosts, Specific Suppressors: Sparse Autoencoder Steering of Medical Vision-Language Models
- 2026LEAD: Layer-wise Expert-aligned Decoding for Faithful Radiology Report Generation