# Region Attention Transformer for Medical Image Restoration

arXiv: https://arxiv.org/abs/2407.09268 · HTML: https://mlsanity.com/paper/2407.09268/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 32 references that are in the map.
- First posted on arXiv 12 Jul 2024.

## Builds on (most cited first, 20 of 20)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021)
- [Enhanced Deep Residual Networks for Single Image Super-Resolution](https://mlsanity.com/paper/1707.02921/) (2017)
- [Image Super-Resolution Using Very Deep Residual Channel Attention Networks](https://mlsanity.com/paper/1807.02758/) (2018)
- [Image Super-Resolution Using Deep Convolutional Networks](https://mlsanity.com/paper/1501.00092/) (2014)
- [Simple Baselines for Image Restoration](https://mlsanity.com/paper/2204.04676/) (2022)
- [Recovering Realistic Texture in Image Super-resolution by Deep Spatial Feature Transform](https://mlsanity.com/paper/1804.02815/) (2018)
- [Vision Transformer with Deformable Attention](https://mlsanity.com/paper/2201.00520/) (2022)
- [Low-Dose CT with a Residual Encoder-Decoder Convolutional Neural Network (RED-CNN)](https://mlsanity.com/paper/1702.00288/) (2017)
- [Low Dose CT Image Denoising Using a Generative Adversarial Network with Wasserstein Distance and Perceptual Loss](https://mlsanity.com/paper/1708.00961/) (2017)
- [Explicit Sparse Transformer: Concentrated Attention Through Explicit Selection](https://mlsanity.com/paper/1912.11637/) (2019)
- [Brain MRI Super Resolution Using 3D Deep Densely Connected Neural Networks](https://mlsanity.com/paper/1801.02728/) (2018)
- [CTformer: Convolution-free Token2Token Dilated Vision Transformer for Low-dose CT Denoising](https://mlsanity.com/paper/2202.13517/) (2022)
- [EDCNN: Edge enhancement-based Densely Connected Network with Compound Loss for Low-Dose CT Denoising](https://mlsanity.com/paper/2011.00139/) (2020)
- [Spach Transformer: Spatial and Channel-wise Transformer Based on Local and Global Self-attentions for PET Image Denoising](https://mlsanity.com/paper/2209.03300/) (2022)
- [Low-dose CT denoising with convolutional neural network](https://mlsanity.com/paper/1610.00321/) (2016)
- [3D Segmentation Guided Style-based Generative Adversarial Networks for PET Synthesis](https://mlsanity.com/paper/2205.08887/) (2022)
- [DRMC: A Generalist Model with Dynamic Routing for Multi-Center PET Image Synthesis](https://mlsanity.com/paper/2307.05249/) (2023)

## Built on by (most cited first, 5 of 5)

- [TAT: Task-Adaptive Transformer for All-in-One Medical Image Restoration](https://mlsanity.com/paper/2512.14550/) (2025)
- [Leveraging Multiphase CT for Quality Enhancement of Portal Venous CT: Utility for Pancreas Segmentation](https://mlsanity.com/paper/2501.14013/) (2025)
- [Pathology Image Restoration via Mixture of Prompts](https://mlsanity.com/paper/2503.12399/) (2025)
- [MARR: Decoupling Policy, Execution, and Calibration for All-in-One Medical Image Restoration](https://mlsanity.com/paper/2609.06645/) (2026)
- [UniH3: Unifying Hierarchical Homogeneity and Heterogeneity for All-in-One Medical Image Restoration](https://mlsanity.com/paper/2609.11156/) (2026)

## Abstract (arXiv)

Transformer-based methods have demonstrated impressive results in medical image restoration, attributed to the multi-head self-attention (MSA) mechanism in the spatial dimension. However, the majority of existing Transformers conduct attention within fixed and coarsely partitioned regions (e.g. the entire image or fixed patches), resulting in interference from irrelevant regions and fragmentation of continuous image content. To overcome these challenges, we introduce a novel Region Attention Transformer (RAT) that utilizes a region-based multi-head self-attention mechanism (R-MSA). The R-MSA dynamically partitions the input image into non-overlapping semantic regions using the robust Segment Anything Model (SAM) and then performs self-attention within these regions. This region partitioning is more flexible and interpretable, ensuring that only pixels from similar semantic regions complement each other, thereby eliminating interference from irrelevant regions. Moreover, we introduce a focal region loss to guide our model to adaptively focus on recovering high-difficulty regions. Extensive experiments demonstrate the effectiveness of RAT in various medical image restoration tasks, including PET image synthesis, CT image denoising, and pathological image super-resolution. Code is available at hrefhttps://github.com/Yaziwel/Region-Attention-Transformer-for-Medical-Image-Restoration.githttps://github.com/RAT.