# Image-to-Image MLP-mixer for Image Reconstruction

arXiv: https://arxiv.org/abs/2202.02018 · HTML: https://mlsanity.com/paper/2202.02018/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 28 references that are in the map.
- First posted on arXiv 4 Feb 2022.

## Builds on (most cited first, 19 of 19)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [MLP-Mixer: An all-MLP Architecture for Vision](https://mlsanity.com/paper/2105.01601/) (2021)
- [Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising](https://mlsanity.com/paper/1608.03981/) (2016)
- [Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentation](https://mlsanity.com/paper/2105.05537/) (2021)
- [Deep Image Prior](https://mlsanity.com/paper/1711.10925/) (2017)
- [fastMRI: An Open Dataset and Benchmarks for Accelerated MRI](https://mlsanity.com/paper/1811.08839/) (2018)
- [Deep Convolutional Neural Network for Inverse Problems in Imaging](https://mlsanity.com/paper/1611.03679/) (2016)
- [Pay Attention to MLPs](https://mlsanity.com/paper/2105.08050/) (2021)
- [ResMLP: Feedforward networks for image classification with data-efficient training](https://mlsanity.com/paper/2105.03404/) (2021)
- [Unprocessing Images for Learned Raw Denoising](https://mlsanity.com/paper/1811.11127/) (2018)
- [End-to-End Variational Networks for Accelerated MRI Reconstruction](https://mlsanity.com/paper/2004.06688/) (2020)
- [CycleMLP: A MLP-like Architecture for Dense Prediction](https://mlsanity.com/paper/2107.10224/) (2021)
- [Deep Decoder: Concise Image Representations from Untrained Non-convolutional Networks](https://mlsanity.com/paper/1810.03982/) (2018)
- [A Review of Convolutional Neural Networks for Inverse Problems in Imaging](https://mlsanity.com/paper/1710.04011/) (2017)
- [Advancing machine learning for MR image reconstruction with an open competition: Overview of the 2019 fastMRI challenge](https://mlsanity.com/paper/2001.02518/) (2020)
- [Denoising and Regularization via Exploiting the Structural Bias of Convolutional Generators](https://mlsanity.com/paper/1910.14634/) (2019)
- [MixerGAN: An MLP-Based Architecture for Unpaired Image-to-Image Translation](https://mlsanity.com/paper/2105.14110/) (2021)

## Built on by (most cited first, 9 of 9)

- [Zero-Shot Noise2Noise: Efficient Image Denoising without any Data](https://mlsanity.com/paper/2303.11253/) (2023)
- [BALF: Simple and Efficient Blur Aware Local Feature Detector](https://mlsanity.com/paper/2211.14731/) (2022)
- [Scaling Laws For Deep Learning Based Image Reconstruction](https://mlsanity.com/paper/2209.13435/) (2022)
- [Full-resolution MLPs Empower Medical Dense Prediction](https://mlsanity.com/paper/2311.16707/) (2023)
- [LoFi: Neural Local Fields for Scalable Image Reconstruction](https://mlsanity.com/paper/2411.04995/) (2024)
- [Computationally Efficient 3D MRI Reconstruction with Adaptive MLP](https://mlsanity.com/paper/2301.08868/) (2023)
- [Fully-fused Multi-Layer Perceptrons on Intel Data Center GPUs](https://mlsanity.com/paper/2403.17607/) (2024)
- [GAMA-IR: Global Additive Multidimensional Averaging for Fast Image Restoration](https://mlsanity.com/paper/2404.00807/) (2024)
- [A self-supervised learning approach for denoising autoregressive models with additive noise: finite and infinite variance cases](https://mlsanity.com/paper/2508.12970/) (2025)

## Abstract (arXiv)

Neural networks are highly effective tools for image reconstruction problems such as denoising and compressive sensing. To date, neural networks for image reconstruction are almost exclusively convolutional. The most popular architecture is the U-Net, a convolutional network with a multi-resolution architecture. In this work, we show that a simple network based on the multi-layer perceptron (MLP)-mixer enables state-of-the art image reconstruction performance without convolutions and without a multi-resolution architecture, provided that the training set and the size of the network are moderately large. Similar to the original MLP-mixer, the image-to-image MLP-mixer is based exclusively on MLPs operating on linearly-transformed image patches. Contrary to the original MLP-mixer, we incorporate structure by retaining the relative positions of the image patches. This imposes an inductive bias towards natural images which enables the image-to-image MLP-mixer to learn to denoise images based on fewer examples than the original MLP-mixer. Moreover, the image-to-image MLP-mixer requires fewer parameters to achieve the same denoising performance than the U-Net and its parameters scale linearly in the image resolution instead of quadratically as for the original MLP-mixer. If trained on a moderate amount of examples for denoising, the image-to-image MLP-mixer outperforms the U-Net by a slight margin. It also outperforms the vision transformer tailored for image reconstruction and classical un-trained methods such as BM3D, making it a very effective tool for image reconstruction problems.