mlsanityData to 14 Oct 2026

Multi-scale Hierarchical Vision Transformer with Cascaded Attention Decoding for Medical Image Segmentation

arXiv 2303.16892 ↗First posted 29 Mar 2023cs.CVcited by 34

Not on the map. Why?

Abstract (arXiv)

Transformers have shown great success in medical image segmentation. However, transformers may exhibit a limited generalization ability due to the underlying single-scale self-attention (SA) mechanism. In this paper, we address this issue by introducing a Multi-scale hiERarchical vIsion Transformer (MERIT) backbone network, which improves the generalizability of the model by computing SA at multiple scales. We also incorporate an attention-based decoder, namely Cascaded Attention Decoding (CASCADE), for further refinement of multi-stage features generated by MERIT. Finally, we introduce an effective multi-stage feature mixing loss aggregation (MUTATION) method for better model training via implicit ensembling. Our experiments on two widely used medical image segmentation benchmarks (i.e., Synapse Multi-organ, ACDC) demonstrate the superior performance of MERIT over state-of-the-art methods. Our MERIT architecture and MUTATION loss aggregation can be used with downstream medical image and semantic segmentation tasks.

Key facts. Cited by 34 papers in the map (data through 14 Oct 2026). Builds on 26 of its 29 references that are in the map. First posted on arXiv 29 Mar 2023.

Builds on

26 of 29 references.

References · 26

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  4. 2017Squeeze-and-Excitation Networkscited by 4,789
  5. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
  6. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,940
  7. 2018CBAM: Convolutional Block Attention Modulecited by 2,477
  8. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,962
  9. 2021TransUNet: Transformers Make Strong Encoders for Medical Image Segmentationcited by 1,558
  10. 2018UNet++: A Nested U-Net Architecture for Medical Image Segmentationcited by 1,525
  11. 2018Attention U-Net: Learning Where to Look for the Pancreascited by 1,435
  12. 2021Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentationcited by 1,222
14 more
  1. 2021PVT v2: Improved Baselines with Pyramid Vision Transformercited by 751
  2. 2022MaxViT: Multi-Axis Vision Transformercited by 476
  3. 2020PraNet: Parallel Reverse Attention Network for Polyp Segmentationcited by 460
  4. 2021Conditional Positional Encodings for Vision Transformerscited by 406
  5. 2020UNet 3+: A Full-Scale Connected UNet for Medical Image Segmentationcited by 332
  6. 2016SCA-CNN: Spatial and Channel-wise Attention in Convolutional Networks for Image Captioningcited by 219
  7. 2021Polyp-PVT: Polyp Segmentation with Pyramid Vision Transformerscited by 159
  8. 2021DS-TransUNet:Dual Swin Transformer U-Net for Medical Image Segmentationcited by 126
  9. 2018Reverse Attention for Salient Object Detectioncited by 113
  10. 2021MISSFormer: An Effective Medical Image Segmentation Transformercited by 103
  11. 2021Mixed Transformer U-Net For Medical Image Segmentationcited by 95
  12. 2022Stepwise Feature Fusion: Local Guides Globalcited by 56
  13. 2022Class-Aware Adversarial Transformers for Medical Image Segmentationcited by 33
  14. 2020DC-UNet: Rethinking the U-Net Architecture with Dual Channel Efficient CNN for Medical Images Segmentationcited by 28

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 6; 2024: 10; 2025: 13; 2026: 5.

34 papers.

Sort:

By area:

  1. 2024EMCAD: Efficient Multi-scale Convolutional Attention Decoding for Medical Image Segmentationcited by 64
  2. 2024Unleashing the Potential of SAM for Medical Adaptation via Hierarchical Decodingcited by 48
  3. 2023Loss Functions in the Era of Semantic Segmentation: A Survey and Outlookcited by 34
  4. 2024Cross-Modal Conditioned Reconstruction for Language-guided Medical Image Segmentationcited by 28
  5. 2023G-CASCADE: Efficient Cascaded Graph Convolutional Decoding for 2D Medical Image Segmentationcited by 21
  6. 2024HiDiff: Hybrid Diffusion Framework for Medical Image Segmentationcited by 13
  7. 2023MIST: Medical Image Segmentation Transformer with Convolutional Attention Mixing (CAM) Decodercited by 12
  8. 2025MK-UNet: Multi-kernel Lightweight CNN for Medical Image Segmentationcited by 12
  9. 2023Attention Mechanisms in Medical Image Segmentation: A Surveycited by 9
  10. 2025Autoregressive Medical Image Segmentation via Next-Scale Mask Predictioncited by 4
  11. 2025PraNet-V2: Dual-Supervised Reverse Attention for Medical Image Segmentationcited by 4
  12. 2023Medical Image Segmentation via Sparse Coding Decodercited by 3
22 more
  1. 2024AgileFormer: Spatially Agile Transformer UNet for Medical Image Segmentationcited by 3
  2. 2024PP-SAM: Perturbed Prompts for Robust Adaptation of Segment Anything Model for Polyp Segmentationcited by 3
  3. 2023AI-SAM: Automatic and Interactive Segment Anything Modelcited by 2
  4. 2025PG-SAM: Prior-Guided SAM with Medical for Multi-organ Segmentationcited by 2
  5. 2025Pixel-wise Modulated Dice Loss for Medical Image Segmentationcited by 2
  6. 2024Multi-dimension Transformer with Attention-based Filtering for Medical Image Segmentationcited by 1
  7. 2024SACNet: A Spatially Adaptive Convolution Network for 2D Multi-organ Medical Segmentationcited by 1
  8. 2024Continuous and complete liver vessel segmentation with graph-attention guided diffusioncited by 1
  9. 2025Advanced Brain Tumor Segmentation Using EMCAD: Efficient Multi-scale Convolutional Attention Decodingcited by 1
  10. 2025Dual Interaction Network with Cross-Image Attention for Medical Image Segmentationcited by 1
  11. 2025LoMix: Learnable Weighted Multi-Scale Logits Mixing for Medical Image Segmentationcited by 1
  12. 2024An Effective UNet Using Feature Interaction and Fusion for Organ Segmentation in Medical Imagecited by 0
  13. 2025Advancing Oyster Phenotype Segmentation with Multi-Network Ensemble and Multi-Scale mechanismcited by 0
  14. 2025Uncertainty-Based Ensemble Learning in CMR Semantic Segmentationcited by 0
  15. 2025MambaCAFU: Hybrid Multi-Scale and Multi-Attention Model with Mamba-Based Fusion for Medical Image Segmentationcited by 0
  16. 2025SA2Net: Scale-Adaptive Structure-Affinity Transformation for Spine Segmentation from Ultrasound Volume Projection Imagingcited by 0
  17. 2025SAMora: Enhancing SAM through Hierarchical Self-Supervised Pre-Training for Medical Imagescited by 0
  18. 2026Gated Differential Linear Attention: A Linear-Time Decoder for High-Fidelity Medical Segmentationcited by 0
  19. 2026Decoding Matters: Efficient Mamba-Based Decoder with Distribution-Aware Deep Supervision for Medical Image Segmentationcited by 0
  20. 2026APRIL-MedSeg: A Modular Medical Image Segmentation Toolbox Embracing Modern Paradigmscited by 0
  21. 2026Lightweight Polyp Segmentation via a Gain-Aware Prediction-Space Recursive Controllercited by 0
  22. 2026HPR-SAM: Hierarchical Probabilistic Representation Learning for Prompt-free SAM-based Medical Image Segmentationcited by 0