# OSRT: Omnidirectional Image Super-Resolution with Distortion-aware Transformer

arXiv: https://arxiv.org/abs/2302.03453 · HTML: https://mlsanity.com/paper/2302.03453/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 51 references that are in the map.
- First posted on arXiv 7 Feb 2023.

## Builds on (most cited first, 28 of 28)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://mlsanity.com/paper/1609.04802/) (2016)
- [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021)
- [Enhanced Deep Residual Networks for Single Image Super-Resolution](https://mlsanity.com/paper/1707.02921/) (2017)
- [Deformable Convolutional Networks](https://mlsanity.com/paper/1703.06211/) (2017)
- [Image Super-Resolution Using Very Deep Residual Channel Attention Networks](https://mlsanity.com/paper/1807.02758/) (2018)
- [ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks](https://mlsanity.com/paper/1809.00219/) (2018)
- [Image Super-Resolution Using Deep Convolutional Networks](https://mlsanity.com/paper/1501.00092/) (2014)
- [Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data](https://mlsanity.com/paper/2107.10833/) (2021)
- [Pre-Trained Image Processing Transformer](https://mlsanity.com/paper/2012.00364/) (2020)
- [Deformable ConvNets v2: More Deformable, Better Results](https://mlsanity.com/paper/1811.11168/) (2018)
- [Learning Continuous Image Representation with Local Implicit Image Function](https://mlsanity.com/paper/2012.09161/) (2020)
- [Designing a Practical Degradation Model for Deep Blind Image Super-Resolution](https://mlsanity.com/paper/2103.14006/) (2021)
- [EDVR: Video Restoration with Enhanced Deformable Convolutional Networks](https://mlsanity.com/paper/1905.02716/) (2019)
- [Activating More Pixels in Image Super-Resolution Transformer](https://mlsanity.com/paper/2205.04437/) (2022)
- [Accelerating the Super-Resolution Convolutional Neural Network](https://mlsanity.com/paper/1608.00367/) (2016)
- [BasicVSR++: Improving Video Super-Resolution with Enhanced Propagation and Alignment](https://mlsanity.com/paper/2104.13371/) (2021)
- [BasicVSR: The Search for Essential Components in Video Super-Resolution and Beyond](https://mlsanity.com/paper/2012.02181/) (2020)
- [Vision Transformer with Deformable Attention](https://mlsanity.com/paper/2201.00520/) (2022)
- [Single Image Super-Resolution via a Holistic Attention Network](https://mlsanity.com/paper/2008.08767/) (2020)
- [TDAN: Temporally Deformable Alignment Network for Video Super-Resolution](https://mlsanity.com/paper/1812.02898/) (2018)
- [RankSRGAN: Generative Adversarial Networks with Ranker for Image Super-Resolution](https://mlsanity.com/paper/1908.06382/) (2019)
- [Improving Image Restoration by Revisiting Global Information Aggregation](https://mlsanity.com/paper/2112.04491/) (2021)
- [Blueprint Separable Residual Network for Efficient Image Super-Resolution](https://mlsanity.com/paper/2205.05996/) (2022)
- [Deep Learning for Omnidirectional Vision: A Survey and New Perspectives](https://mlsanity.com/paper/2205.10468/) (2022)
- [SphereSR: 360deg Image Super-Resolution with Arbitrary Projection via Continuous Spherical Image Representation](https://mlsanity.com/paper/2112.06536/) (2021)
- [Super-resolution of Omnidirectional Images Using Adversarial Learning](https://mlsanity.com/paper/1908.04297/) (2019)

## Built on by (most cited first, 26 of 26)

- [Egocentric Whole-Body Motion Capture with FisheyeViT and Diffusion-Based Motion Refinement](https://mlsanity.com/paper/2311.16495/) (2023)
- [CAMixerSR: Only Details Need More "Attention"](https://mlsanity.com/paper/2402.19289/) (2024)
- [A Survey of Representation Learning, Optimization Strategies, and Applications for Omnidirectional Vision](https://mlsanity.com/paper/2502.10444/) (2025)
- [Minimalist and High-Quality Panoramic Imaging with PSF-aware Transformers](https://mlsanity.com/paper/2306.12992/) (2023)
- [TULIP: Transformer for Upsampling of LiDAR Point Clouds](https://mlsanity.com/paper/2312.06733/) (2023)
- [Neural Radiance Field-based Visual Rendering: A Comprehensive Review](https://mlsanity.com/paper/2404.00714/) (2024)
- [OPDN: Omnidirectional Position-aware Deformable Network for Omnidirectional Image Super-Resolution](https://mlsanity.com/paper/2304.13471/) (2023)
- [Structural Attention: Rethinking Transformer for Unpaired Medical Image Synthesis](https://mlsanity.com/paper/2406.18967/) (2024)
- [Occlusion-Aware Seamless Segmentation](https://mlsanity.com/paper/2407.02182/) (2024)
- [ResVR: Joint Rescaling and Viewport Rendering of Omnidirectional Images](https://mlsanity.com/paper/2404.16825/) (2024)
- [Geometric Distortion Guided Transformer for Omnidirectional Image Super-Resolution](https://mlsanity.com/paper/2406.10869/) (2024)
- [Estimating Depth of Monocular Panoramic Image with Teacher-Student Model Fusing Equirectangular and Spherical Representations](https://mlsanity.com/paper/2405.16858/) (2024)
- [PanoWorld: Towards Spatial Supersensing in 360^circ Panorama World](https://mlsanity.com/paper/2605.13169/) (2026)
- [PAC-FNO: Parallel-Structured All-Component Fourier Neural Operators for Recognizing Low-Quality Images](https://mlsanity.com/paper/2402.12721/) (2024)
- [PriOr-Flow: Enhancing Primitive Panoramic Optical Flow with Orthogonal View](https://mlsanity.com/paper/2506.23897/) (2025)
- [RealOSR: Latent Guidance Boosts Diffusion-based Real-world Omnidirectional Image Super-Resolutions](https://mlsanity.com/paper/2412.09646/) (2024)
- [Fast Omni-Directional Image Super-Resolution: Adapting the Implicit Image Function with Pixel and Semantic-Wise Spherical Geometric Priors](https://mlsanity.com/paper/2502.05902/) (2025)
- [Spatio-Temporal Distortion Aware Omnidirectional Video Super-Resolution](https://mlsanity.com/paper/2410.11506/) (2024)
- [Adaptive Convolutional Neural Network for Image Super-resolution](https://mlsanity.com/paper/2402.15704/) (2024)
- [Learning High-Quality Navigation and Zooming on Omnidirectional Images in Virtual Reality](https://mlsanity.com/paper/2405.00351/) (2024)
- [360VFI: A Dataset and Benchmark for Omnidirectional Video Frame Interpolation](https://mlsanity.com/paper/2407.14066/) (2024)
- [360-Degree Full-view Image Segmentation by Spherical Convolution compatible with Large-scale Planar Pre-trained Models](https://mlsanity.com/paper/2507.09216/) (2025)
- [Panoramic Scene Understanding: A Survey from Distortion-Aware Engineering to Sphere-Native Modeling](https://mlsanity.com/paper/2606.27745/) (2026)
- [360^circ High-Resolution Depth Estimation via Uncertainty-aware Structural Knowledge Transfer](https://mlsanity.com/paper/2304.07967/) (2023)
- [D2R2OSR: Degradation-Disentangled Representation for Real-World Omnidirectional Image Super-Resolution](https://mlsanity.com/paper/2606.29314/) (2026)
- [Seek to Segment: Active Perception for Panoramic Referring Segmentation](https://mlsanity.com/paper/2607.02497/) (2026)

## Abstract (arXiv)

Omnidirectional images (ODIs) have obtained lots of research interest for immersive experiences. Although ODIs require extremely high resolution to capture details of the entire scene, the resolutions of most ODIs are insufficient. Previous methods attempt to solve this issue by image super-resolution (SR) on equirectangular projection (ERP) images. However, they omit geometric properties of ERP in the degradation process, and their models can hardly generalize to real ERP images. In this paper, we propose Fisheye downsampling, which mimics the real-world imaging process and synthesizes more realistic low-resolution samples. Then we design a distortion-aware Transformer (OSRT) to modulate ERP distortions continuously and self-adaptively. Without a cumbersome process, OSRT outperforms previous methods by about 0.2dB on PSNR. Moreover, we propose a convenient data augmentation strategy, which synthesizes pseudo ERP images from plain images. This simple strategy can alleviate the over-fitting problem of large networks and significantly boost the performance of ODISR. Extensive experiments have demonstrated the state-of-the-art performance of our OSRT. Codes and models will be available at https://github.com/Fanghua-Yu/OSRT.