mlsanityData to 14 Oct 2026

Prototype-Driven and Multi-Expert Integrated Multi-Modal MR Brain Tumor Image Segmentation

arXiv 2307.12180 ↗First posted 22 Jul 2023eess.IV cs.CV cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

For multi-modal magnetic resonance (MR) brain tumor image segmentation, current methods usually directly extract the discriminative features from input images for tumor sub-region category determination and localization. However, the impact of information aliasing caused by the mutual inclusion of tumor sub-regions is often ignored. Moreover, existing methods usually do not take tailored efforts to highlight the single tumor sub-region features. To this end, a multi-modal MR brain tumor segmentation method with tumor prototype-driven and multi-expert integration is proposed. It could highlight the features of each tumor sub-region under the guidance of tumor prototypes. Specifically, to obtain the prototypes with complete information, we propose a mutual transmission mechanism to transfer different modal features to each other to address the issues raised by insufficient information on single-modal features. Furthermore, we devise a prototype-driven feature representation and fusion method with the learned prototypes, which implants the prototypes into tumor features and generates corresponding activation maps. With the activation maps, the sub-region features consistent with the prototype category can be highlighted. A key information enhancement and fusion strategy with multi-expert integration is designed to further improve the segmentation performance. The strategy can integrate the features from different layers of the extra feature extraction network and the features highlighted by the prototypes. Experimental results on three competition brain tumor segmentation datasets prove the superiority of the proposed method.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 36 of its 72 references that are in the map. First posted on arXiv 22 Jul 2023.

Builds on

36 of 72 references.

References · 36

  1. 2017Attention Is All You Needcited by 60,361
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  4. 2016V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentationcited by 2,292
  5. 2021TransUNet: Transformers Make Strong Encoders for Medical Image Segmentationcited by 1,558
  6. 20163D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotationcited by 1,476
  7. 2018Attention U-Net: Learning Where to Look for the Pancreascited by 1,435
  8. 2021Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentationcited by 1,222
  9. 2021UNETR: Transformers for 3D Medical Image Segmentationcited by 959
  10. 2022Swin UNETR: Swin Transformers for Semantic Segmentation of Brain Tumors in MRI Imagescited by 830
  11. 2017Road Extraction by Deep Residual U-Netcited by 333
  12. 20183D MRI brain tumor segmentation using autoencoder regularizationcited by 328
24 more
  1. 2021TransBTS: Multimodal Brain Tumor Segmentation Using Transformercited by 265
  2. 2016Efficient Multi-Scale 3D CNN with Fully Connected CRF for Accurate Brain Lesion Segmentationcited by 263
  3. 2019Models Genesis: Generic Autodidactic Models for 3D Medical Image Analysiscited by 159
  4. 2015Brain Tumor Segmentation with Deep Neural Networkscited by 137
  5. 2022mmFormer: Multimodal Medical Transformer for Incomplete Multimodal Learning of Brain Tumor Segmentationcited by 95
  6. 2019Hetero-Modal Variational Encoder-Decoder for Joint Modality Completion and Segmentationcited by 68
  7. 2020Robust Multimodal Brain Tumor Segmentation via Feature Disentanglement and Gated Fusioncited by 68
  8. 2023M3AE: Multimodal Representation Learning for Brain Tumor Segmentation with Missing Modalitiescited by 56
  9. 2021Modality-aware Mutual Learning for Multi-modal Medical Image Segmentationcited by 47
  10. 2021ACN: Adversarial Co-training Network for Brain Tumor Segmentation with Missing Modalitiescited by 43
  11. 2021Latent Correlation Representation Learning for Brain Tumor Segmentation with Missing MRI Modalitiescited by 42
  12. 2023Diff-UNet: A Diffusion Embedded Network for Volumetric Segmentationcited by 41
  13. 2018Automatic Brain Tumor Segmentation using Convolutional Neural Networks with Test-Time Augmentationcited by 36
  14. 2022NestedFormer: Nested Modality-Aware Transformer for Brain Tumor Segmentationcited by 34
  15. 2020KiU-Net: Overcomplete Convolutional Architectures for Biomedical Image and Volumetric Segmentationcited by 32
  16. 2022SMU-Net: Style matching U-Net for brain tumor segmentation with missing modalitiescited by 31
  17. 2022CKD-TransBTS: Clinical Knowledge-Driven Hybrid Transformer with Modality-Correlated Cross-Attention for Brain Tumor Segmentationcited by 24
  18. 2022Cross-Modality Deep Feature Learning for Brain Tumor Segmentationcited by 22
  19. 2019Multi-step Cascaded Networks for Brain Tumor Segmentationcited by 11
  20. 2021A Tri-attention Fusion Guided Multi-modal Segmentation Networkcited by 10
  21. 2022TransBTSV2: Towards Better and More Efficient Volumetric Segmentation of Medical Imagescited by 7
  22. 2019OctopusNet: A Deep Learning Segmentation Network for Multi-modal Medical Imagescited by 5
  23. 2022Category Guided Attention Network for Brain Tumor Segmentation in MRIcited by 3
  24. 2022HMRNet: High and Multi-Resolution Network with Bidirectional Feature Calibration for Brain Structure Segmentation in Radiotherapycited by 1

Built on by

Not cited yet.