# CKD-TransBTS: Clinical Knowledge-Driven Hybrid Transformer with Modality-Correlated Cross-Attention for Brain Tumor Segmentation

arXiv: https://arxiv.org/abs/2207.07370 · HTML: https://mlsanity.com/paper/2207.07370/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 57 references that are in the map.
- First posted on arXiv 15 Jul 2022.

## Builds on (most cited first, 30 of 36)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation](https://mlsanity.com/paper/2102.04306/) (2021)
- [EfficientNetV2: Smaller Models and Faster Training](https://mlsanity.com/paper/2104.00298/) (2021)
- [Transformers in Vision: A Survey](https://mlsanity.com/paper/2101.01169/) (2021)
- [UNETR: Transformers for 3D Medical Image Segmentation](https://mlsanity.com/paper/2103.10504/) (2021)
- [Swin UNETR: Swin Transformers for Semantic Segmentation of Brain Tumors in MRI Images](https://mlsanity.com/paper/2201.01266/) (2022)
- [A Survey on Visual Transformer](https://mlsanity.com/paper/2012.12556/) (2020)
- [UNet++: Redesigning Skip Connections to Exploit Multiscale Features in Image Segmentation](https://mlsanity.com/paper/1912.05074/) (2019)
- [The RSNA-ASNR-MICCAI BraTS 2021 Benchmark on Brain Tumor Segmentation and Radiogenomic Classification](https://mlsanity.com/paper/2107.02314/) (2021)
- [Multimodal Learning with Transformers: A Survey](https://mlsanity.com/paper/2206.06488/) (2022)
- [Early Convolutions Help Transformers See Better](https://mlsanity.com/paper/2106.14881/) (2021)
- [Road Extraction by Deep Residual U-Net](https://mlsanity.com/paper/1711.10684/) (2017)
- [Attention Gated Networks: Learning to Leverage Salient Regions in Medical Images](https://mlsanity.com/paper/1808.08114/) (2018)
- [TransBTS: Multimodal Brain Tumor Segmentation Using Transformer](https://mlsanity.com/paper/2103.04430/) (2021)
- [Transformers in Medical Imaging: A Survey](https://mlsanity.com/paper/2201.09873/) (2022)
- [CoTr: Efficiently Bridging CNN and Transformer for 3D Medical Image Segmentation](https://mlsanity.com/paper/2103.03024/) (2021)
- [Confidence Calibration and Predictive Uncertainty Estimation for Deep Medical Image Segmentation](https://mlsanity.com/paper/1911.13273/) (2019)
- [Understanding and Mitigating Annotation Bias in Facial Expression Recognition](https://mlsanity.com/paper/2108.08504/) (2021)
- [A Robust Volumetric Transformer for Accurate 3D Tumor Segmentation](https://mlsanity.com/paper/2111.13300/) (2021)
- [Optimized U-Net for Brain Tumor Segmentation](https://mlsanity.com/paper/2110.03352/) (2021)
- [Extending nn-UNet for brain tumor segmentation](https://mlsanity.com/paper/2112.04653/) (2021)
- [A deep learning model integrating FCNNs and CRFs for brain tumor segmentation](https://mlsanity.com/paper/1702.04528/) (2017)
- [Deep Learning Based Brain Tumor Segmentation: A Survey](https://mlsanity.com/paper/2007.09479/) (2020)
- [One-pass Multi-task Networks with Cross-task Guided Attention for Brain Tumor Segmentation](https://mlsanity.com/paper/1906.01796/) (2019)
- [Cross-Modality Deep Feature Learning for Brain Tumor Segmentation](https://mlsanity.com/paper/2201.02356/) (2022)

## Built on by (most cited first, 24 of 24)

- [A Recent Survey of Vision Transformers for Medical Image Segmentation](https://mlsanity.com/paper/2312.00634/) (2023)
- [Shapley Values-enabled Progressive Pseudo Bag Augmentation for Whole Slide Image Classification](https://mlsanity.com/paper/2312.05490/) (2023)
- [Multimodal Data Integration for Precision Oncology: Challenges and Future Directions](https://mlsanity.com/paper/2406.19611/) (2024)
- [Rethinking Brain Tumor Segmentation from the Frequency Domain Perspective](https://mlsanity.com/paper/2506.10142/) (2025)
- [Brain Tumor Classification using Vision Transformer with Selective Cross-Attention Mechanism and Feature Calibration](https://mlsanity.com/paper/2406.17670/) (2024)
- [FDiff-Fusion:Denoising diffusion fusion network based on fuzzy learning for 3D medical image segmentation](https://mlsanity.com/paper/2408.02075/) (2024)
- [mAIstro: an open-source multi-agentic system for automated end-to-end development of radiomics and deep learning models for medical imaging](https://mlsanity.com/paper/2505.03785/) (2025)
- [Brain Tumor Segmentation Based on Deep Learning, Attention Mechanisms, and Energy-Based Uncertainty Prediction](https://mlsanity.com/paper/2401.00587/) (2023)
- [Multimodal Information Interaction for Medical Image Segmentation](https://mlsanity.com/paper/2404.16371/) (2024)
- [Automatic Quantification of Serial PET/CT Images for Pediatric Hodgkin Lymphoma Patients Using a Longitudinally-Aware Segmentation Network](https://mlsanity.com/paper/2404.08611/) (2024)
- [Advancing Efficient Brain Tumor Multi-Class Classification -- New Insights from the Vision Mamba Model in Transfer Learning](https://mlsanity.com/paper/2410.21872/) (2024)
- [Multimodal Fusion at Three Tiers: Physics-Driven Data Generation and Vision-Language Guidance for Brain Tumor Segmentation](https://mlsanity.com/paper/2507.09966/) (2025)
- [Prototype-Driven and Multi-Expert Integrated Multi-Modal MR Brain Tumor Image Segmentation](https://mlsanity.com/paper/2307.12180/) (2023)
- [Modality-Aware and Shift Mixer for Multi-modal Brain Tumor Segmentation](https://mlsanity.com/paper/2403.02074/) (2024)
- [Hybrid Multihead Attentive Unet-3D for Brain Tumor Segmentation](https://mlsanity.com/paper/2405.13304/) (2024)
- [Decoupling Feature Representations of Ego and Other Modalities for Incomplete Multi-modal Brain Tumor Segmentation](https://mlsanity.com/paper/2408.08708/) (2024)
- [multiPI-TransBTS: A Multi-Path Learning Framework for Brain Tumor Image Segmentation Based on Multi-Physical Information](https://mlsanity.com/paper/2409.12167/) (2024)
- [Selective Complementary Feature Fusion and Modal Feature Compression Interaction for Brain Tumor Segmentation](https://mlsanity.com/paper/2503.16149/) (2025)
- [Knowledge-Guided Brain Tumor Segmentation via Synchronized Visual-Semantic-Topological Prior Fusion](https://mlsanity.com/paper/2507.08574/) (2025)
- [A Systematic Review on the Generative AI Applications in Human Medical Genomics](https://mlsanity.com/paper/2508.20275/) (2025)
- [Beyond CLIP: Knowledge-Enhanced Multimodal Transformers for Cross-Modal Alignment in Diabetic Retinopathy Diagnosis](https://mlsanity.com/paper/2512.19663/) (2025)
- [ReFRM3D: A Radiomics-enhanced Fused Residual Multiparametric 3D Network with Multi-Scale Feature Fusion for Glioma Characterization](https://mlsanity.com/paper/2512.22570/) (2025)
- [Degradation-Aware Blur-Segmentation of Brain Tumor](https://mlsanity.com/paper/2605.15671/) (2026)
- [FSS-UBrain: Multi-region Few-Shot Brain Tumor MRI Segmentation](https://mlsanity.com/paper/2609.32273/) (2026)

## Abstract (arXiv)

Brain tumor segmentation (BTS) in magnetic resonance image (MRI) is crucial for brain tumor diagnosis, cancer management and research purposes. With the great success of the ten-year BraTS challenges as well as the advances of CNN and Transformer algorithms, a lot of outstanding BTS models have been proposed to tackle the difficulties of BTS in different technical aspects. However, existing studies hardly consider how to fuse the multi-modality images in a reasonable manner. In this paper, we leverage the clinical knowledge of how radiologists diagnose brain tumors from multiple MRI modalities and propose a clinical knowledge-driven brain tumor segmentation model, called CKD-TransBTS. Instead of directly concatenating all the modalities, we re-organize the input modalities by separating them into two groups according to the imaging principle of MRI. A dual-branch hybrid encoder with the proposed modality-correlated cross-attention block (MCCA) is designed to extract the multi-modality image features. The proposed model inherits the strengths from both Transformer and CNN with the local feature representation ability for precise lesion boundaries and long-range feature extraction for 3D volumetric images. To bridge the gap between Transformer and CNN features, we propose a Trans&CNN Feature Calibration block (TCFC) in the decoder. We compare the proposed model with five CNN-based models and six transformer-based models on the BraTS 2021 challenge dataset. Extensive experiments demonstrate that the proposed model achieves state-of-the-art brain tumor segmentation performance compared with all the competitors.