mlsanityData to 7 Oct 2026

Multi-modal Brain Tumor Segmentation via Missing Modality Synthesis and Modality-level Attention Fusion

arXiv 2203.04586 ↗First posted 9 Mar 2022eess.IV cs.CVcited by 2

Not on the map. Why?

Abstract (arXiv)

Multi-modal magnetic resonance (MR) imaging provides great potential for diagnosing and analyzing brain gliomas. In clinical scenarios, common MR sequences such as T1, T2 and FLAIR can be obtained simultaneously in a single scanning process. However, acquiring contrast enhanced modalities such as T1ce requires additional time, cost, and injection of contrast agent. As such, it is clinically meaningful to develop a method to synthesize unavailable modalities which can also be used as additional inputs to downstream tasks (e.g., brain tumor segmentation) for performance enhancing. In this work, we propose an end-to-end framework named Modality-Level Attention Fusion Network (MAF-Net), wherein we innovatively conduct patchwise contrastive learning for extracting multi-modal latent features and dynamically assigning attention weights to fuse different modalities. Through extensive experiments on BraTS2020, our proposed MAF-Net is found to yield superior T1ce synthesis performance (SSIM of 0.8879 and PSNR of 22.78) and accurate brain tumor segmentation (mean Dice scores of 67.9%, 41.8% and 88.0% on segmenting the tumor core, enhancing tumor and whole tumor).

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 17 of its 26 references that are in the map. First posted on arXiv 9 Mar 2022.

Builds on

17 of 26 references.

References · 17

  1. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  2. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  3. 2017Squeeze-and-Excitation Networkscited by 4,763
  4. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  5. 2020Contrastive Learning for Unpaired Image-to-Image Translationcited by 637
  6. 2021Multi-Modal Fusion Transformer for End-to-End Autonomous Drivingcited by 372
  7. 2019DRIT++: Diverse Image-to-Image Translation via Disentangled Representationscited by 244
  8. 2019Missing MRI Pulse Sequence Synthesis using Multi-Modal Generative Adversarial Networkcited by 98
  9. 2018Translating and Segmenting Multimodal Medical Volumes with Cycle- and Shape-Consistency Generative Adversarial Networkcited by 97
  10. 2020Robust Multimodal Brain Tumor Segmentation via Feature Disentanglement and Gated Fusioncited by 67
  11. 2018HyperDense-Net: A hyper-densely connected CNN for multi-modal image segmentationcited by 66
  12. 2019Unsupervised Domain Adaptation via Disentangled Representations: Application to Cross-Modality Liver Segmentationcited by 50
5 more
  1. 2021Latent Correlation Representation Learning for Brain Tumor Segmentation with Missing MRI Modalitiescited by 42
  2. 2018Brain Tumor Segmentation using an Ensemble of 3D U-Nets and Overall Survival Prediction using Radiomic Featurescited by 33
  3. 2018MRI Tumor Segmentation with Densely Connected 3D CNNcited by 16
  4. 2021Modality Completion via Gaussian Process Prior Variational Autoencoders for Multi-Modal Glioma Segmentationcited by 16
  5. 2020Multi-Modality Generative Adversarial Networks with Tumor Consistency Loss for Brain MR Image Synthesiscited by 12

Built on by

Cited by year

Citing papers per year: 2022: 2.

2 papers.

  1. 2022MouseGAN++: Unsupervised Disentanglement and Contrastive Representation for Multiple MRI Modalities Synthesis and Structural Segmentation of Mouse Braincited by 6
  2. 2022DS3-Net: Difficulty-perceived Common-to-T1ce Semi-Supervised Multimodal MRI Synthesis Networkcited by 1