# ViTBIS: Vision Transformer for Biomedical Image Segmentation

arXiv: https://arxiv.org/abs/2201.05920 · HTML: https://mlsanity.com/paper/2201.05920/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 44 references that are in the map.
- First posted on arXiv 15 Jan 2022.

## Builds on (most cited first, 30 of 32)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation](https://mlsanity.com/paper/2102.04306/) (2021)
- [UNet++: A Nested U-Net Architecture for Medical Image Segmentation](https://mlsanity.com/paper/1807.10165/) (2018)
- [3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation](https://mlsanity.com/paper/1606.06650/) (2016)
- [Attention U-Net: Learning Where to Look for the Pancreas](https://mlsanity.com/paper/1804.03999/) (2018)
- [Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentation](https://mlsanity.com/paper/2105.05537/) (2021)
- [UNETR: Transformers for 3D Medical Image Segmentation](https://mlsanity.com/paper/2103.10504/) (2021)
- [Image Transformer](https://mlsanity.com/paper/1802.05751/) (2018)
- [Identifying the Best Machine Learning Algorithms for Brain Tumor Segmentation, Progression Assessment, and Overall Survival Prediction in the BRATS Challenge](https://mlsanity.com/paper/1811.02629/) (2018)
- [A large annotated medical image dataset for the development and evaluation of segmentation algorithms](https://mlsanity.com/paper/1902.09063/) (2019)
- [3D MRI brain tumor segmentation using autoencoder regularization](https://mlsanity.com/paper/1810.11654/) (2018)
- [Medical Transformer: Gated Axial-Attention for Medical Image Segmentation](https://mlsanity.com/paper/2102.10662/) (2021)
- [Attention Gated Networks: Learning to Leverage Salient Regions in Medical Images](https://mlsanity.com/paper/1808.08114/) (2018)
- [TransBTS: Multimodal Brain Tumor Segmentation Using Transformer](https://mlsanity.com/paper/2103.04430/) (2021)
- [TransFuse: Fusing Transformers and CNNs for Medical Image Segmentation](https://mlsanity.com/paper/2102.08005/) (2021)
- [CoTr: Efficiently Bridging CNN and Transformer for 3D Medical Image Segmentation](https://mlsanity.com/paper/2103.03024/) (2021)
- [Local Relation Networks for Image Recognition](https://mlsanity.com/paper/1904.11491/) (2019)
- [Multi-scale self-guided attention for medical image segmentation](https://mlsanity.com/paper/1906.02849/) (2019)
- [Optimization for Medical Image Segmentation: Theory and Practice when evaluating with Dice Score or Jaccard Index](https://mlsanity.com/paper/2010.13499/) (2020)
- [Domain Adaptive Relational Reasoning for 3D Multi-Organ Segmentation](https://mlsanity.com/paper/2005.09120/) (2020)
- [RA-UNet: A hybrid deep attention-aware network to extract liver and tumor in CT scans](https://mlsanity.com/paper/1811.01328/) (2018)
- [Automatic 3D Cardiovascular MR Segmentation with Densely-Connected Volumetric ConvNets](https://mlsanity.com/paper/1708.00573/) (2017)
- [Uncertainty Quantification using Variational Inference for Biomedical Image Segmentation](https://mlsanity.com/paper/2008.07588/) (2020)
- [DMSANet: Dual Multi Scale Attention Network](https://mlsanity.com/paper/2106.08382/) (2021)
- [Monocular Depth Estimation Using Multi Scale Neural Network And Feature Fusion](https://mlsanity.com/paper/2009.09934/) (2020)

## Built on by (most cited first, 5 of 5)

- [Vision Transformers in Medical Computer Vision -- A Contemplative Retrospection](https://mlsanity.com/paper/2203.15269/) (2022)
- [A Comprehensive Survey of Transformers for Computer Vision](https://mlsanity.com/paper/2211.06004/) (2022)
- [Recent Progress in Transformer-based Medical Image Analysis](https://mlsanity.com/paper/2208.06643/) (2022)
- [Attention Mechanisms in Medical Image Segmentation: A Survey](https://mlsanity.com/paper/2305.17937/) (2023)
- [Transformers Meet Visual Learning Understanding: A Comprehensive Review](https://mlsanity.com/paper/2203.12944/) (2022)

## Abstract (arXiv)

In this paper, we propose a novel network named Vision Transformer for Biomedical Image Segmentation (ViTBIS). Our network splits the input feature maps into three parts with 1× 1, 3× 3 and 5× 5 convolutions in both encoder and decoder. Concat operator is used to merge the features before being fed to three consecutive transformer blocks with attention mechanism embedded inside it. Skip connections are used to connect encoder and decoder transformer blocks. Similarly, transformer blocks and multi scale architecture is used in decoder before being linearly projected to produce the output segmentation map. We test the performance of our network using Synapse multi-organ segmentation dataset, Automated cardiac diagnosis challenge dataset, Brain tumour MRI segmentation dataset and Spleen CT segmentation dataset. Without bells and whistles, our network outperforms most of the previous state of the art CNN and transformer based models using Dice score and the Hausdorff distance as the evaluation metrics.