# Xformer: Hybrid X-Shaped Transformer for Image Denoising

arXiv: https://arxiv.org/abs/2303.06440 · HTML: https://mlsanity.com/paper/2303.06440/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- Builds on 46 of its 67 references that are in the map.
- First posted on arXiv 11 Mar 2023.

## Builds on (most cited first, 30 of 46)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions](https://mlsanity.com/paper/2102.12122/) (2021)
- [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021)
- [Restormer: Efficient Transformer for High-Resolution Image Restoration](https://mlsanity.com/paper/2111.09881/) (2021)
- [Enhanced Deep Residual Networks for Single Image Super-Resolution](https://mlsanity.com/paper/1707.02921/) (2017)
- [Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising](https://mlsanity.com/paper/1608.03981/) (2016)
- [Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network](https://mlsanity.com/paper/1609.05158/) (2016)
- [Self-Attention with Relative Position Representations](https://mlsanity.com/paper/1803.02155/) (2018)
- [Pre-Trained Image Processing Transformer](https://mlsanity.com/paper/2012.00364/) (2020)
- [Uformer: A General U-Shaped Transformer for Image Restoration](https://mlsanity.com/paper/2106.03106/) (2021)
- [Multi-Stage Progressive Image Restoration](https://mlsanity.com/paper/2102.02808/) (2021)
- [Twins: Revisiting the Design of Spatial Attention in Vision Transformers](https://mlsanity.com/paper/2104.13840/) (2021)
- [FFDNet: Toward a Fast and Flexible Solution for CNN based Image Denoising](https://mlsanity.com/paper/1710.04026/) (2017)
- [Plug-and-Play Image Restoration with Deep Denoiser Prior](https://mlsanity.com/paper/2008.13751/) (2020)
- [Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution](https://mlsanity.com/paper/1704.03915/) (2017)
- [Learning Deep CNN Denoiser Prior for Image Restoration](https://mlsanity.com/paper/1704.03264/) (2017)
- [XCiT: Cross-Covariance Image Transformers](https://mlsanity.com/paper/2106.09681/) (2021)
- [Learning Enriched Features for Real Image Restoration and Enhancement](https://mlsanity.com/paper/2003.06792/) (2020)
- [MAXIM: Multi-Axis MLP for Image Processing](https://mlsanity.com/paper/2201.02973/) (2022)
- [Toward Convolutional Blind Denoising of Real Photographs](https://mlsanity.com/paper/1807.04686/) (2018)
- [Residual Non-local Attention Networks for Image Restoration](https://mlsanity.com/paper/1903.10082/) (2019)
- [Co-Scale Conv-Attentional Image Transformers](https://mlsanity.com/paper/2104.06399/) (2021)
- [CycleISP: Real Image Restoration via Improved Data Synthesis](https://mlsanity.com/paper/2003.07761/) (2020)

## Built on by (most cited first, 30 of 37)

- [MambaIRv2: Attentive State Space Restoration](https://mlsanity.com/paper/2411.15269/) (2024)
- [DreamClear: High-Capacity Real-World Image Restoration with Privacy-Safe Dataset Curation](https://mlsanity.com/paper/2410.18666/) (2024)
- [Masked Image Training for Generalizable Deep Image Denoising](https://mlsanity.com/paper/2303.13132/) (2023)
- [Restore-RWKV: Efficient and Effective Medical Image Restoration with RWKV](https://mlsanity.com/paper/2407.11087/) (2024)
- [From Zero to Detail: Deconstructing Ultra-High-Definition Image Restoration from Progressive Spectral Perspective](https://mlsanity.com/paper/2503.13165/) (2025)
- [The Tenth NTIRE 2025 Image Denoising Challenge Report](https://mlsanity.com/paper/2504.12276/) (2025)
- [BIGbench: A Unified Benchmark for Evaluating Multi-dimensional Social Biases in Text-to-Image Models](https://mlsanity.com/paper/2407.15240/) (2024)
- [Sharing Key Semantics in Transformer Makes Efficient Image Restoration](https://mlsanity.com/paper/2405.20008/) (2024)
- [Reversing Flow for Image Restoration](https://mlsanity.com/paper/2506.16961/) (2025)
- [FlashSVD: Memory-Efficient Inference with Streaming for Low-Rank Models](https://mlsanity.com/paper/2508.01506/) (2025)
- [Lightweight Medical Image Restoration via Integrating Reliable Lesion-Semantic Driven Prior](https://mlsanity.com/paper/2504.11286/) (2025)
- [Complementary Advantages: Exploiting Cross-Field Frequency Correlation for NIR-Assisted Image Denoising](https://mlsanity.com/paper/2412.16645/) (2024)
- [NIR-Assisted Image Denoising: A Selective Fusion Approach and A Real-World Benchmark Dataset](https://mlsanity.com/paper/2404.08514/) (2024)
- [Self-Calibrated Variance-Stabilizing Transformations for Real-World Image Denoising](https://mlsanity.com/paper/2407.17399/) (2024)
- [Learning to Translate Noise for Robust Image Denoising](https://mlsanity.com/paper/2412.04727/) (2024)
- [Key-Graph Transformer for Image Restoration](https://mlsanity.com/paper/2402.02634/) (2024)
- [ATD: Improved Transformer with Adaptive Token Dictionary for Image Restoration](https://mlsanity.com/paper/2603.02581/) (2026)
- [TReX- Reusing Vision Transformer's Attention for Efficient Xbar-based Computing](https://mlsanity.com/paper/2408.12742/) (2024)
- [Fractal-IR: A Unified Framework for Efficient and Scalable Image Restoration](https://mlsanity.com/paper/2503.17825/) (2025)
- [Beyond the Ground Truth: Enhanced Supervision for Image Restoration](https://mlsanity.com/paper/2512.03932/) (2025)
- [Spiking Pyramid Wavelet Transformation for High-efficient and Low-energy Image Restoration](https://mlsanity.com/paper/2606.18644/) (2026)
- [Bi-Level Spatial and Channel-aware Transformer for Learned Image Compression](https://mlsanity.com/paper/2408.03842/) (2024)
- [Classification-Denoising Networks](https://mlsanity.com/paper/2410.03505/) (2024)
- [Freqformer: Frequency-Domain Transformer for 3-D Reconstruction and Quantification of Human Retinal Vasculature](https://mlsanity.com/paper/2411.11189/) (2024)
- [Hierarchical Information Flow for Generalized Efficient Image Restoration](https://mlsanity.com/paper/2411.18588/) (2024)
- [Personalized Generative Low-light Image Denoising and Enhancement](https://mlsanity.com/paper/2412.14327/) (2024)
- [StreamFlow: Theory, Algorithm, and Implementation for High-Efficiency Rectified Flow Generation](https://mlsanity.com/paper/2511.22009/) (2025)
- [KOSS: Kalman-Optimal Selective State Spaces for Long-Term Sequence Modeling](https://mlsanity.com/paper/2512.16723/) (2025)
- [Progressive Split Mamba: Effective State Space Modelling for Image Restoration](https://mlsanity.com/paper/2603.09171/) (2026)
- [The Third Challenge on Image Denoising at NTIRE 2026: Methods and Results](https://mlsanity.com/paper/2606.16031/) (2026)

## Abstract (arXiv)

In this paper, we present a hybrid X-shaped vision Transformer, named Xformer, which performs notably on image denoising tasks. We explore strengthening the global representation of tokens from different scopes. In detail, we adopt two types of Transformer blocks. The spatial-wise Transformer block performs fine-grained local patches interactions across tokens defined by spatial dimension. The channel-wise Transformer block performs direct global context interactions across tokens defined by channel dimension. Based on the concurrent network structure, we design two branches to conduct these two interaction fashions. Within each branch, we employ an encoder-decoder architecture to capture multi-scale features. Besides, we propose the Bidirectional Connection Unit (BCU) to couple the learned representations from these two branches while providing enhanced information fusion. The joint designs make our Xformer powerful to conduct global information modeling in both spatial and channel dimensions. Extensive experiments show that Xformer, under the comparable model complexity, achieves state-of-the-art performance on the synthetic and real-world image denoising tasks. We also provide code and models at https://github.com/gladzhang/Xformer.