mlsanityData to 14 Oct 2026

On the Utility of Foundation Models for Fast MRI: Vision-Language-Guided Image Reconstruction

arXiv 2511.19641 ↗First posted 24 Nov 2025cs.CV cs.AIcited by 4

On the map: Multimodal LLMs / vision-language models → Medical vision-language models

Abstract (arXiv)

Purpose: To investigate whether a vision-language foundation model can enhance undersampled MRI reconstruction by providing high-level contextual information beyond conventional priors. Methods: We proposed a semantic distribution-guided reconstruction framework that uses a pre-trained vision-language foundation model to encode both the reconstructed image and auxiliary information into high-level semantic features. A contrastive objective aligns the reconstructed representation with the target semantic distribution, ensuring consistency with high-level perceptual cues. The proposed objective works with various deep learning-based reconstruction methods and can flexibly incorporate semantic priors from multimodal sources. To test the effectiveness of these semantic priors, we evaluated reconstruction results guided by priors derived from either image-only or image-language auxiliary information. Results: Experiments on knee and brain datasets demonstrate that semantic priors from images preserve fine anatomical structures and achieve superior perceptual quality, as reflected in lower LPIPS values, higher Tenengrad scores, and improved scores in the reader study, compared with conventional regularization. The image-language information further expands the semantic distribution and enables high-level control over reconstruction attributes. Across all evaluations, the contrastive objective consistently guided the reconstructed features toward the desired semantic distributions while maintaining data fidelity, demonstrating the effectiveness of the proposed optimization framework. Conclusion: The study highlights that vision-language foundation models can improve undersampled MRI reconstruction through semantic-space optimization.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Medical vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 55 of its 97 references that are in the map. First posted on arXiv 24 Nov 2025.

Builds on

55 of 97 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Vision-Language Models for Medical Report Generation and Visual Question Answering: A Review4 Mar

Other references · 54

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  5. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  6. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  7. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  8. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  9. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  10. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,864
  11. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
  12. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,898
38 more
  1. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,329
  2. 2020Implicit Neural Representations with Periodic Activation Functionscited by 2,238
  3. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 2,031
  4. 2024Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Thinkcited by 703
  5. 2017The Perception-Distortion Tradeoffcited by 587
  6. 2022MedCLIP: Contrastive Learning from Unpaired Medical Images and Textcited by 570
  7. 2023Medical SAM Adapter: Adapting Segment Anything Model for Medical Image Segmentationcited by 490
  8. 2018fastMRI: An Open Dataset and Benchmarks for Accelerated MRIcited by 470
  9. 2024Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generationcited by 457
  10. 2025Reconstruction vs. Generation: Taming Optimization Dilemma in Latent Diffusion Modelscited by 393
  11. 2017MoDL: Model Based Deep Learning Architecture for Inverse Problemscited by 387
  12. 2017Learning a Variational Network for Reconstruction of Accelerated MRI Datacited by 342
  13. 2021Score-based diffusion models for accelerated MRIcited by 311
  14. 2016Adversarial Training Methods for Semi-Supervised Text Classificationcited by 280
  15. 2023Multimodal Foundation Models: From Specialists to General-Purpose Assistantscited by 247
  16. 2023Adapted Large Language Models Can Outperform Medical Experts in Clinical Text Summarizationcited by 246
  17. 2017Image reconstruction by domain transform manifold learningcited by 232
  18. 2019On instabilities of deep learning in image reconstruction - Does AI come at a cost?cited by 212
  19. 2023Solving Inverse Problems with Latent Diffusion Models via Hard Data Consistencycited by 182
  20. 2021NeRP: Implicit Neural Representation Learning with Prior Embedding for Sparsely Sampled Image Reconstructioncited by 146
  21. 2022Adaptive Diffusion Priors for Accelerated MRI Reconstructioncited by 118
  22. 2019Implicit Semantic Data Augmentation for Deep Networkscited by 106
  23. 2024Segment Anything Model for Medical Image Segmentation: Current Applications and Future Directionscited by 97
  24. 2019Time-Dependent Deep Image Prior for Dynamic MRIcited by 70
  25. 2022Swin Transformer for Fast MRIcited by 56
  26. 2024SD-DiT: Unleashing the Power of Self-supervised Discrimination in Diffusion Transformercited by 49
  27. 2024BrainSegFounder: Towards 3D Foundation Models for Neuroimage Segmentationcited by 39
  28. 2022Ranking Info Noise Contrastive Estimation: Boosting Contrastive Learning via Ranked Positivescited by 37
  29. 2020Multi-label Contrastive Predictive Codingcited by 32
  30. 2025Triad: Vision Foundation Model for 3D Magnetic Resonance Imagingcited by 24
  31. 2023IMJENSE: Scan-specific Implicit Representation for Joint Coil Sensitivity and Image Estimation in Parallel MRIcited by 23
  32. 2023Diffusion Modeling with Domain-conditioned Prior Guidance for Accelerated MRI and qMRI Reconstructioncited by 22
  33. 2025MRI-CORE: A Foundation Model for Magnetic Resonance Imagingcited by 15
  34. 2025No Alignment Needed for Generation: Learning Linearly Separable Representations in Diffusion Modelscited by 8
  35. 2025A Trust-Guided Approach to MR Image Reconstruction with Side Informationcited by 7
  36. 2023Self-Supervised Federated Learning for Fast MR Imagingcited by 5
  37. 2025Generative Autoregressive Transformers for Model-Agnostic Federated MRI Reconstructioncited by 5
  38. 2018A Deep Error Correction Network for Compressed Sensing MRIcited by 3

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026Accelerating Stroke MRI with Diffusion Probabilistic Models through Large-Scale Pre-training and Target-Specific Fine-Tuningcited by 0
  2. 2026HyDAR-Pano3D: A Hybrid Disentangled Anatomical Recovery Framework for Panoramic-to-3D Reconstructioncited by 0
  3. 2026Data-Efficient Networks for Multi-Contrast MRI Reconstruction based on a Generalized Content/Style Priorcited by 0
  4. 2026Learning Subject-Specific Anatomical Representations via Manifold Expansion: Application to Accelerated Multi-Contrast MRIcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024LDPM: Towards undersampled MRI reconstruction with MR-VAE and Latent Diffusion Prior
  2. 2026Vision-Language Model-Guided Deep Unrolling Enables Personalized, Fast MRI
  3. 2025Leveraging Vision-Language Foundation Models to Reveal Hidden Image-Attribute Relationships in Medical Imaging
  4. 2026Sparse Autoencoders for Interpretable Medical Image Representation Learning
  5. 2026SynMind: Reducing Semantic Hallucination in fMRI-Based Image Reconstruction
  6. 2022Adapting Pretrained Vision-Language Foundational Models to Medical Imaging Domains
  7. 2025Lesion-Aware Post-Training of Latent Diffusion Models for Synthesizing Diffusion MRI from CT Perfusion
  8. 2026Visual Instruction-Finetuned Language Model for Versatile Brain MR Image Tasks