# Brain Image Synthesis with Unsupervised Multivariate Canonical CSCℓ4Net

arXiv: https://arxiv.org/abs/2103.11587 · HTML: https://mlsanity.com/paper/2103.11587/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 41 references that are in the map.
- First posted on arXiv 22 Mar 2021.

## Builds on (most cited first, 14 of 14)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Moment Matching for Multi-Source Domain Adaptation](https://mlsanity.com/paper/1812.01754/) (2018)
- [High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs](https://mlsanity.com/paper/1711.11585/) (2017)
- [Few-Shot Unsupervised Image-to-Image Translation](https://mlsanity.com/paper/1905.01723/) (2019)
- [Data augmentation using learned transformations for one-shot medical image segmentation](https://mlsanity.com/paper/1902.09383/) (2019)
- [Mode Seeking Generative Adversarial Networks for Diverse Image Synthesis](https://mlsanity.com/paper/1903.05628/) (2019)
- [Translating and Segmenting Multimodal Medical Volumes with Cycle- and Shape-Consistency Generative Adversarial Network](https://mlsanity.com/paper/1802.09655/) (2018)
- [Unsupervised Multi-Target Domain Adaptation: An Information Theoretic Approach](https://mlsanity.com/paper/1810.11547/) (2018)
- [Adversarial Defense by Stratified Convolutional Sparse Coding](https://mlsanity.com/paper/1812.00037/) (2018)
- [Degenerative Adversarial NeuroImage Nets: Generating Images that Mimic Disease Progression](https://mlsanity.com/paper/1907.02787/) (2019)
- [Structured Local Optima in Sparse Blind Deconvolution](https://mlsanity.com/paper/1806.00338/) (2018)
- [Multi-scale GANs for Memory-efficient Generation of High Resolution Medical Images](https://mlsanity.com/paper/1907.01376/) (2019)
- [DOTE: Dual cOnvolutional filTer lEarning for Super-Resolution and Cross-Modality Synthesis in MRI](https://mlsanity.com/paper/1706.04954/) (2017)
- [Joint registration and synthesis using a probabilistic model for alignment of MRI and histological sections](https://mlsanity.com/paper/1801.05284/) (2018)

## Built on by (most cited first, 3 of 3)

- [GuidedMix-Net: Semi-supervised Semantic Segmentation by Using Labeled Images as Reference](https://mlsanity.com/paper/2112.14015/) (2021)
- [Adaptive PromptNet For Auxiliary Glioma Diagnosis without Contrast-Enhanced MRI](https://mlsanity.com/paper/2211.07966/) (2022)
- [DMVFC: Deep Learning Based Functionally Consistent Tractography Fiber Clustering Using Multimodal Diffusion MRI and Functional MRI](https://mlsanity.com/paper/2510.24770/) (2025)

## Abstract (arXiv)

Recent advances in neuroscience have highlighted the effectiveness of multi-modal medical data for investigating certain pathologies and understanding human cognition. However, obtaining full sets of different modalities is limited by various factors, such as long acquisition times, high examination costs and artifact suppression. In addition, the complexity, high dimensionality and heterogeneity of neuroimaging data remains another key challenge in leveraging existing randomized scans effectively, as data of the same modality is often measured differently by different machines. There is a clear need to go beyond the traditional imaging-dependent process and synthesize anatomically specific target-modality data from a source input. In this paper, we propose to learn dedicated features that cross both intre- and intra-modal variations using a novel CSCℓ4Net. Through an initial unification of intra-modal data in the feature maps and multivariate canonical adaptation, CSCℓ4Net facilitates feature-level mutual transformation. The positive definite Riemannian manifold-penalized data fidelity term further enables CSCℓ4Net to reconstruct missing measurements according to transformed features. Finally, the maximization ℓ4-norm boils down to a computationally efficient optimization problem. Extensive experiments validate the ability and robustness of our CSCℓ4Net compared to the state-of-the-art methods on multiple datasets.