# CoT-MISR:Marrying Convolution and Transformer for Multi-Image Super-Resolution

arXiv: https://arxiv.org/abs/2303.06548 · HTML: https://mlsanity.com/paper/2303.06548/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 59 references that are in the map.
- First posted on arXiv 12 Mar 2023.

## Builds on (most cited first, 25 of 25)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021)
- [Image Super-Resolution Using Very Deep Residual Channel Attention Networks](https://mlsanity.com/paper/1807.02758/) (2018)
- [Deep Mutual Learning](https://mlsanity.com/paper/1706.00384/) (2017)
- [Accelerating the Super-Resolution Convolutional Neural Network](https://mlsanity.com/paper/1608.00367/) (2016)
- [Learning Texture Transformer Network for Image Super-Resolution](https://mlsanity.com/paper/2006.04139/) (2020)
- [DualGAN: Unsupervised Dual Learning for Image-to-Image Translation](https://mlsanity.com/paper/1704.02510/) (2017)
- [Deep Back-Projection Networks For Super-Resolution](https://mlsanity.com/paper/1803.02735/) (2018)
- [Real-Time Video Super-Resolution with Spatio-Temporal Networks and Motion Compensation](https://mlsanity.com/paper/1611.05250/) (2016)
- [Feedback Network for Image Super-Resolution](https://mlsanity.com/paper/1903.09814/) (2019)
- [Dual Learning for Machine Translation](https://mlsanity.com/paper/1611.00179/) (2016)
- [Unsupervised Image Super-Resolution using Cycle-in-Cycle Generative Adversarial Networks](https://mlsanity.com/paper/1809.00437/) (2018)
- [Closed-loop Matters: Dual Regression Networks for Single Image Super-Resolution](https://mlsanity.com/paper/2003.07018/) (2020)
- [Improving Multispectral Pedestrian Detection by Addressing Modality Imbalance Problems](https://mlsanity.com/paper/2008.03043/) (2020)
- [Wide Activation for Efficient and Accurate Image Super-Resolution](https://mlsanity.com/paper/1808.08718/) (2018)
- [HighRes-net: Recursive Fusion for Multi-Frame Super-Resolution of Satellite Imagery](https://mlsanity.com/paper/2002.06460/) (2020)
- [Dual Supervised Learning](https://mlsanity.com/paper/1707.00415/) (2017)
- [DeepSUM: Deep neural network for Super-resolution of Unregistered Multitemporal images](https://mlsanity.com/paper/1907.06490/) (2019)
- [Multi-image Super Resolution of Remotely Sensed Images using Residual Feature Attention Deep Neural Networks](https://mlsanity.com/paper/2007.03107/) (2020)
- [Permutation invariance and uncertainty in multitemporal image super-resolution](https://mlsanity.com/paper/2105.12409/) (2021)
- [Super-Resolution of PROBA-V Images Using Convolutional Neural Networks](https://mlsanity.com/paper/1907.01821/) (2019)
- [Deep Learning for Multiple-Image Super-Resolution](https://mlsanity.com/paper/1903.00440/) (2019)
- [3DSRnet: Video Super-resolution using 3D Convolutional Neural Networks](https://mlsanity.com/paper/1812.09079/) (2018)
- [Unsupervised Degradation Learning for Single Image Super-Resolution](https://mlsanity.com/paper/1812.04240/) (2018)
- [Image quality assessment for determining efficacy and limitations of Super-Resolution Convolutional Neural Network (SRCNN)](https://mlsanity.com/paper/1905.05373/) (2019)

## Abstract (arXiv)

As a method of image restoration, image super-resolution has been extensively studied at first. How to transform a low-resolution image to restore its high-resolution image information is a problem that researchers have been exploring. In the early physical transformation methods, the high-resolution pictures generated by these methods always have a serious problem of missing information, and the edges and details can not be well recovered. With the development of hardware technology and mathematics, people begin to use in-depth learning methods for image super-resolution tasks, from direct in-depth learning models, residual channel attention networks, bi-directional suppression networks, to tr networks with transformer network modules, which have gradually achieved good results. In the research of multi-graph super-resolution, thanks to the establishment of multi-graph super-resolution dataset, we have experienced the evolution from convolution model to transformer model, and the quality of super-resolution has been continuously improved. However, we find that neither pure convolution nor pure tr network can make good use of low-resolution image information. Based on this, we propose a new end-to-end CoT-MISR network. CoT-MISR network makes up for local and global information by using the advantages of convolution and tr. The validation of dataset under equal parameters shows that our CoT-MISR network has reached the optimal score index.