# Multi-Modality Conditioned Variational U-Net for Field-of-View Extension in Brain Diffusion MRI

arXiv: https://arxiv.org/abs/2409.13846 · HTML: https://mlsanity.com/paper/2409.13846/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 65 references that are in the map.
- First posted on arXiv 20 Sep 2024.

## Builds on (most cited first, 11 of 11)

- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [A Variational U-Net for Conditional Appearance and Shape Generation](https://mlsanity.com/paper/1804.04694/) (2018)
- [Spatial Broadcast Decoder: A Simple Architecture for Learning Disentangled Representations in VAEs](https://mlsanity.com/paper/1901.07017/) (2019)
- [TractSeg - Fast and accurate white matter tract segmentation](https://mlsanity.com/paper/1805.07103/) (2018)
- [DDM2: Self-Supervised Diffusion MRI Denoising with Generative Diffusion Models](https://mlsanity.com/paper/2302.03018/) (2023)
- [Patch2Self: Denoising Diffusion MRI with Self-Supervised Learning](https://mlsanity.com/paper/2011.01355/) (2020)
- [SDnDTI: Self-supervised deep learning-based denoising for diffusion tensor MRI](https://mlsanity.com/paper/2111.07220/) (2021)
- [Spatio-Angular Convolutions for Super-resolution in Diffusion MRI](https://mlsanity.com/paper/2306.00854/) (2023)
- [Longitudinal diffusion MRI analysis using Segis-Net: a single-step deep-learning framework for simultaneous segmentation and registration](https://mlsanity.com/paper/2012.14230/) (2020)
- [Field-of-View Extension for Brain Diffusion MRI via Deep Generative Models](https://mlsanity.com/paper/2405.03652/) (2024)
- [FOD-Swin-Net: angular super resolution of fiber orientation distribution using a transformer-based deep model](https://mlsanity.com/paper/2402.11775/) (2024)

## Built on by (most cited first, 2 of 2)

- [TractCloud-FOV: Deep Learning-based Robust Tractography Parcellation in Diffusion MRI with Incomplete Field of View](https://mlsanity.com/paper/2502.20637/) (2025)
- [Generalizable field of view extension for diffusion MRI via conditional flow matching](https://mlsanity.com/paper/2610.17401/) (2026)

## Abstract (arXiv)

An incomplete field-of-view (FOV) in diffusion magnetic resonance imaging (dMRI) can severely hinder the volumetric and bundle analyses of whole-brain white matter connectivity. Although existing works have investigated imputing the missing regions using deep generative models, it remains unclear how to specifically utilize additional information from paired multi-modality data and whether this can enhance the imputation quality and be useful for downstream tractography. To fill this gap, we propose a novel framework for imputing dMRI scans in the incomplete part of the FOV by integrating the learned diffusion features in the acquired part of the FOV to the complete brain anatomical structure. We hypothesize that by this design the proposed framework can enhance the imputation performance of the dMRI scans and therefore be useful for repairing whole-brain tractography in corrupted dMRI scans with incomplete FOV. We tested our framework on two cohorts from different sites with a total of 96 subjects and compared it with a baseline imputation method that treats the information from T1w and dMRI scans equally. The proposed framework achieved significant improvements in imputation performance, as demonstrated by angular correlation coefficient (p < 1E-5), and in downstream tractography accuracy, as demonstrated by Dice score (p < 0.01). Results suggest that the proposed framework improved imputation performance in dMRI scans by specifically utilizing additional information from paired multi-modality data, compared with the baseline method. The imputation achieved by the proposed framework enhances whole brain tractography, and therefore reduces the uncertainty when analyzing bundles associated with neurodegenerative.