# Image Super-Resolution with Taylor Expansion Approximation and Large Field Reception

arXiv: https://arxiv.org/abs/2408.00470 · HTML: https://mlsanity.com/paper/2408.00470/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 35 of its 57 references that are in the map.
- First posted on arXiv 1 Aug 2024.

## Builds on (most cited first, 30 of 35)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Xception: Deep Learning with Depthwise Separable Convolutions](https://mlsanity.com/paper/1610.02357/) (2016)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021)
- [Restormer: Efficient Transformer for High-Resolution Image Restoration](https://mlsanity.com/paper/2111.09881/) (2021)
- [Image Super-Resolution Using Very Deep Residual Channel Attention Networks](https://mlsanity.com/paper/1807.02758/) (2018)
- [Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data](https://mlsanity.com/paper/2107.10833/) (2021)
- [Accurate Image Super-Resolution Using Very Deep Convolutional Networks](https://mlsanity.com/paper/1511.04587/) (2015)
- [Designing a Practical Degradation Model for Deep Blind Image Super-Resolution](https://mlsanity.com/paper/2103.14006/) (2021)
- [Plug-and-Play Image Restoration with Deep Denoiser Prior](https://mlsanity.com/paper/2008.13751/) (2020)
- [Sketch-based Manga Retrieval using Manga109 Dataset](https://mlsanity.com/paper/1510.04389/) (2015)
- ["Zero-Shot" Super-Resolution using Deep Internal Learning](https://mlsanity.com/paper/1712.06087/) (2017)
- [Fast, Accurate, and Lightweight Super-Resolution with Cascading Residual Network](https://mlsanity.com/paper/1803.08664/) (2018)
- [Deep Unfolding Network for Image Super-Resolution](https://mlsanity.com/paper/2003.10428/) (2020)
- [Learning a Single Convolutional Super-Resolution Network for Multiple Degradations](https://mlsanity.com/paper/1712.06116/) (2017)
- [Visual Attention Network](https://mlsanity.com/paper/2202.09741/) (2022)
- [Blind Super-Resolution With Iterative Kernel Correction](https://mlsanity.com/paper/1904.03377/) (2019)
- [FLatten Transformer: Vision Transformer using Focused Linear Attention](https://mlsanity.com/paper/2308.00442/) (2023)
- [Unfolding the Alternating Optimization for Blind Super Resolution](https://mlsanity.com/paper/2010.02631/) (2020)
- [Deep Plug-and-Play Super-Resolution for Arbitrary Blur Kernels](https://mlsanity.com/paper/1903.12529/) (2019)
- [Mutual Affine Network for Spatially Variant Kernel Estimation in Blind Image Super-Resolution](https://mlsanity.com/paper/2108.05302/) (2021)
- [Deep Constrained Least Squares for Blind Image Super-Resolution](https://mlsanity.com/paper/2202.07508/) (2022)
- [Multi-scale Attention Network for Single Image Super-Resolution](https://mlsanity.com/paper/2209.14145/) (2022)
- [A Dynamic Residual Self-Attention Network for Lightweight Single Image Super-Resolution](https://mlsanity.com/paper/2112.04488/) (2021)
- [End-to-end Alternating Optimization for Blind Super Resolution](https://mlsanity.com/paper/2105.06878/) (2021)

## Abstract (arXiv)

Self-similarity techniques are booming in blind super-resolution (SR) due to accurate estimation of the degradation types involved in low-resolution images. However, high-dimensional matrix multiplication within self-similarity computation prohibitively consumes massive computational costs. We find that the high-dimensional attention map is derived from the matrix multiplication between Query and Key, followed by a softmax function. This softmax makes the matrix multiplication between Query and Key inseparable, posing a great challenge in simplifying computational complexity. To address this issue, we first propose a second-order Taylor expansion approximation (STEA) to separate the matrix multiplication of Query and Key, resulting in the complexity reduction from O(N2) to O(N). Then, we design a multi-scale large field reception (MLFR) to compensate for the performance degradation caused by STEA. Finally, we apply these two core designs to laboratory and real-world scenarios by constructing LabNet and RealNet, respectively. Extensive experimental results tested on five synthetic datasets demonstrate that our LabNet sets a new benchmark in qualitative and quantitative evaluations. Tested on the RealWorld38 dataset, our RealNet achieves superior visual quality over existing methods. Ablation studies further verify the contributions of STEA and MLFR towards both LabNet and RealNet frameworks.