# Language-Enhanced Generative Modeling for Amyloid PET Synthesis from MRI and Blood Biomarkers

arXiv: https://arxiv.org/abs/2511.02206 · HTML: https://mlsanity.com/paper/2511.02206/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 79 references that are in the map.
- First posted on arXiv 4 Nov 2025.

## Builds on (most cited first, 19 of 19)

- [FiLM: Visual Reasoning with a General Conditioning Layer](https://mlsanity.com/paper/1709.07871/) (2017)
- [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014)
- [Large Language Models Encode Clinical Knowledge](https://mlsanity.com/paper/2212.13138/) (2022)
- [3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation](https://mlsanity.com/paper/1606.06650/) (2016)
- [Generative Adversarial Text to Image Synthesis](https://mlsanity.com/paper/1605.05396/) (2016)
- [A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applications](https://mlsanity.com/paper/2402.07927/) (2024)
- [Med3D: Transfer Learning for 3D Medical Image Analysis](https://mlsanity.com/paper/1904.00625/) (2019)
- [SynthSeg: Segmentation of brain MRI scans of any contrast and resolution without retraining](https://mlsanity.com/paper/2107.09559/) (2021)
- [StackGAN++: Realistic Image Synthesis with Stacked Generative Adversarial Networks](https://mlsanity.com/paper/1710.10916/) (2017)
- [Design Guidelines for Prompt Engineering Text-to-Image Generative Models](https://mlsanity.com/paper/2109.06977/) (2021)
- [Prompt Engineering for Healthcare: Methodologies and Applications](https://mlsanity.com/paper/2304.14670/) (2023)
- [A Transformer-based representation-learning model with unified processing of multimodal input for clinical diagnostics](https://mlsanity.com/paper/2306.00864/) (2023)
- [Learning to synthesise the ageing brain without longitudinal data](https://mlsanity.com/paper/1912.02620/) (2019)
- [Bidirectional Mapping Generative Adversarial Networks for Brain MR to PET Synthesis](https://mlsanity.com/paper/2008.03483/) (2020)
- [Deep Multi-modal Fusion of Image and Non-image Data in Disease Diagnosis and Prognosis: A Review](https://mlsanity.com/paper/2203.15588/) (2022)
- [Data-Centric Foundation Models in Computational Healthcare: A Survey](https://mlsanity.com/paper/2401.02458/) (2024)
- [Fr'echet Radiomic Distance (FRD): A Versatile Metric for Comparing Medical Imaging Datasets](https://mlsanity.com/paper/2412.01496/) (2024)
- [HyperFusion: A Hypernetwork Approach to Multimodal Integration of Tabular and Medical Imaging Data for Predictive Modeling](https://mlsanity.com/paper/2403.13319/) (2024)
- [WASABI: A Metric for Evaluating Morphometric Plausibility of Synthetic Brain MRIs](https://mlsanity.com/paper/2504.21771/) (2025)

## Built on by (most cited first, 1 of 1)

- [Brain-WM: Brain Glioblastoma World Model](https://mlsanity.com/paper/2603.07562/) (2026)

## Abstract (arXiv)

Background: Alzheimer's disease (AD) diagnosis heavily relies on amyloid-beta positron emission tomography (Abeta-PET), which is limited by high cost and limited accessibility. This study explores whether Abeta-PET spatial patterns can be predicted from blood-based biomarkers (BBMs) and MRI scans. Methods: We collected Abeta-PET images, T1-weighted MRI scans, and BBMs from 566 participants. A language-enhanced generative model, driven by a large language model (LLM) and multimodal information fusion, was developed to synthesize PET images. Synthesized images were evaluated for image quality, diagnostic consistency, and clinical applicability within a fully automated diagnostic pipeline. Findings: The synthetic PET images closely resemble real PET scans in both structural details (SSIM = 0.920 +/- 0.003) and regional patterns (Pearson's r = 0.955 +/- 0.007). Diagnostic outcomes using synthetic PET show high agreement with real PET-based diagnoses (accuracy = 0.80). Using synthetic PET, we developed a fully automatic AD diagnostic pipeline integrating PET synthesis and classification. The synthetic PET-based model (AUC = 0.78) outperforms T1-based (AUC = 0.68) and BBM-based (AUC = 0.73) models, while combining synthetic PET and BBMs further improved performance (AUC = 0.79). Ablation analysis supports the advantages of LLM integration and prompt engineering. Interpretation: Our language-enhanced generative model synthesizes realistic PET images, enhancing the utility of MRI and BBMs for Abeta spatial pattern assessment and improving the diagnostic workflow for Alzheimer's disease.