# UltraSR: Spatial Encoding is a Missing Key for Implicit Image Function-based Arbitrary-Scale Super-Resolution

arXiv: https://arxiv.org/abs/2103.12716 · HTML: https://mlsanity.com/paper/2103.12716/

## Key facts

- Cited by 57 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 55 references that are in the map.
- First posted on arXiv 23 Mar 2021.

## Builds on (most cited first, 30 of 40)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Large Scale GAN Training for High Fidelity Natural Image Synthesis](https://mlsanity.com/paper/1809.11096/) (2018)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Occupancy Networks: Learning 3D Reconstruction in Function Space](https://mlsanity.com/paper/1812.03828/) (2018)
- [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://mlsanity.com/paper/1609.04802/) (2016)
- [Enhanced Deep Residual Networks for Single Image Super-Resolution](https://mlsanity.com/paper/1707.02921/) (2017)
- [Deformable Convolutional Networks](https://mlsanity.com/paper/1703.06211/) (2017)
- [On the Spectral Bias of Neural Networks](https://mlsanity.com/paper/1806.08734/) (2018)
- [Image Super-Resolution Using Very Deep Residual Channel Attention Networks](https://mlsanity.com/paper/1807.02758/) (2018)
- [Image Super-Resolution Using Deep Convolutional Networks](https://mlsanity.com/paper/1501.00092/) (2014)
- [Neural Sparse Voxel Fields](https://mlsanity.com/paper/2007.11571/) (2020)
- [Accurate Image Super-Resolution Using Very Deep Convolutional Networks](https://mlsanity.com/paper/1511.04587/) (2015)
- [NeRF++: Analyzing and Improving Neural Radiance Fields](https://mlsanity.com/paper/2010.07492/) (2020)
- [Residual Dense Network for Image Super-Resolution](https://mlsanity.com/paper/1802.08797/) (2018)
- [Convolutional Occupancy Networks](https://mlsanity.com/paper/2003.04618/) (2020)
- [Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervision](https://mlsanity.com/paper/1912.07372/) (2019)
- [GRAF: Generative Radiance Fields for 3D-Aware Image Synthesis](https://mlsanity.com/paper/2007.02442/) (2020)
- [Learning Continuous Image Representation with Local Implicit Image Function](https://mlsanity.com/paper/2012.09161/) (2020)
- [Learning Deep CNN Denoiser Prior for Image Restoration](https://mlsanity.com/paper/1704.03264/) (2017)
- [SAL: Sign Agnostic Learning of Shapes from Raw Data](https://mlsanity.com/paper/1911.10414/) (2019)
- [Local Implicit Grid Representations for 3D Scenes](https://mlsanity.com/paper/2003.08981/) (2020)
- [Sketch-based Manga Retrieval using Manga109 Dataset](https://mlsanity.com/paper/1510.04389/) (2015)
- [Deep Local Shapes: Learning Local SDF Priors for Detailed 3D Reconstruction](https://mlsanity.com/paper/2003.10983/) (2020)
- [Learning Shape Templates with Structured Implicit Functions](https://mlsanity.com/paper/1904.06447/) (2019)
- [NeX: Real-time View Synthesis with Neural Basis Expansion](https://mlsanity.com/paper/2103.05606/) (2021)

## Built on by (most cited first, 30 of 57)

- [Scale-MAE: A Scale-Aware Masked Autoencoder for Multiscale Geospatial Representation Learning](https://mlsanity.com/paper/2212.14532/) (2022)
- [CIPS-3D: A 3D-Aware Generator of GANs Based on Conditionally-Independent Pixel Synthesis](https://mlsanity.com/paper/2110.09788/) (2021)
- [VideoINR: Learning Video Implicit Neural Representation for Continuous Space-Time Super-Resolution](https://mlsanity.com/paper/2206.04647/) (2022)
- [E-NeRV: Expedite Neural Video Representation with Disentangled Spatial-Temporal Context](https://mlsanity.com/paper/2207.08132/) (2022)
- [CiaoSR: Continuous Implicit Attention-in-Attention Network for Arbitrary-Scale Image Super-Resolution](https://mlsanity.com/paper/2212.04362/) (2022)
- [Cascaded Local Implicit Transformer for Arbitrary-Scale Super-Resolution](https://mlsanity.com/paper/2303.16513/) (2023)
- [Pixel to Gaussian: Ultra-Fast Continuous Super-Resolution with 2D Gaussian Modeling](https://mlsanity.com/paper/2503.06617/) (2025)
- [Enhancing Video Super-Resolution via Implicit Resampling-based Alignment](https://mlsanity.com/paper/2305.00163/) (2023)
- [Local Implicit Normalizing Flow for Arbitrary-Scale Image Super-Resolution](https://mlsanity.com/paper/2303.05156/) (2023)
- [Learning Implicit Feature Alignment Function for Semantic Segmentation](https://mlsanity.com/paper/2206.08655/) (2022)
- [Improving Point-based Crowd Counting and Localization Based on Auxiliary Point Guidance](https://mlsanity.com/paper/2405.10589/) (2024)
- [Latent Modulated Function for Computational Optimal Continuous Image Representation](https://mlsanity.com/paper/2404.16451/) (2024)
- [Continuous Remote Sensing Image Super-Resolution based on Context Interaction in Implicit Function Space](https://mlsanity.com/paper/2302.08046/) (2023)
- [Implicit Neural Representation Learning for Hyperspectral Image Super-Resolution](https://mlsanity.com/paper/2112.10541/) (2021)
- [Image Completion with Heterogeneously Filtered Spectral Hints](https://mlsanity.com/paper/2211.03700/) (2022)
- [OPE-SR: Orthogonal Position Encoding for Designing a Parameter-free Upsampling Module in Arbitrary-scale Image Super-Resolution](https://mlsanity.com/paper/2303.01091/) (2023)
- [AnyFlow: Arbitrary Scale Optical Flow with Implicit Neural Representation](https://mlsanity.com/paper/2303.16493/) (2023)
- [Spectral-wise Implicit Neural Representation for Hyperspectral Image Reconstruction](https://mlsanity.com/paper/2312.01061/) (2023)
- [DDMI: Domain-Agnostic Latent Diffusion Models for Synthesizing High-Quality Implicit Neural Representations](https://mlsanity.com/paper/2401.12517/) (2024)
- [Dissecting Arbitrary-scale Super-resolution Capability from Pre-trained Diffusion Generative Models](https://mlsanity.com/paper/2306.00714/) (2023)
- [CycleINR: Cycle Implicit Neural Representation for Arbitrary-Scale Volumetric Super-Resolution of Medical Data](https://mlsanity.com/paper/2404.04878/) (2024)
- [Enhancing Multi-Scale Implicit Learning in Image Super-Resolution with Integrated Positional Encoding](https://mlsanity.com/paper/2112.05756/) (2021)
- [S3Mamba: Arbitrary-Scale Super-Resolution via Scaleable State Space Model](https://mlsanity.com/paper/2411.11906/) (2024)
- [Fourier-enhanced Implicit Neural Fusion Network for Multispectral and Hyperspectral Image Fusion](https://mlsanity.com/paper/2404.15174/) (2024)
- [Revisiting Implicit Neural Representations in Low-Level Vision](https://mlsanity.com/paper/2304.10250/) (2023)
- [Continuous Cross-resolution Remote Sensing Image Change Detection](https://mlsanity.com/paper/2305.14722/) (2023)
- [Boosting Flow-based Generative Super-Resolution Models via Learned Prior](https://mlsanity.com/paper/2403.10988/) (2024)
- [Lightweight Stepless Super-Resolution of Remote Sensing Images via Saliency-Aware Dynamic Routing Strategy](https://mlsanity.com/paper/2210.07598/) (2022)
- [Thera: Aliasing-Free Arbitrary-Scale Super-Resolution with Neural Heat Fields](https://mlsanity.com/paper/2311.17643/) (2023)
- [Neural Gaussian Scale-Space Fields](https://mlsanity.com/paper/2405.20980/) (2024)

## Abstract (arXiv)

The recent success of NeRF and other related implicit neural representation methods has opened a new path for continuous image representation, where pixel values no longer need to be looked up from stored discrete 2D arrays but can be inferred from neural network models on a continuous spatial domain. Although the recent work LIIF has demonstrated that such novel approaches can achieve good performance on the arbitrary-scale super-resolution task, their upscaled images frequently show structural distortion due to the inaccurate prediction of high-frequency textures. In this work, we propose UltraSR, a simple yet effective new network design based on implicit image functions in which we deeply integrated spatial coordinates and periodic encoding with the implicit neural representation. Through extensive experiments and ablation studies, we show that spatial encoding is a missing key toward the next-stage high-performing implicit image function. Our UltraSR sets new state-of-the-art performance on the DIV2K benchmark under all super-resolution scales compared to previous state-of-the-art methods. UltraSR also achieves superior performance on other standard benchmark datasets in which it outperforms prior works in almost all experiments.