# Latent Modulated Function for Computational Optimal Continuous Image Representation

arXiv: https://arxiv.org/abs/2404.16451 · HTML: https://mlsanity.com/paper/2404.16451/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 46 references that are in the map.
- First posted on arXiv 25 Apr 2024.

## Builds on (most cited first, 30 of 35)

- [FiLM: Visual Reasoning with a General Conditioning Layer](https://mlsanity.com/paper/1709.07871/) (2017)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields](https://mlsanity.com/paper/2111.12077/) (2021)
- [Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields](https://mlsanity.com/paper/2103.13415/) (2021)
- [Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network](https://mlsanity.com/paper/1609.04802/) (2016)
- [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021)
- [pixelNeRF: Neural Radiance Fields from One or Few Images](https://mlsanity.com/paper/2012.02190/) (2020)
- [Enhanced Deep Residual Networks for Single Image Super-Resolution](https://mlsanity.com/paper/1707.02921/) (2017)
- [Image Super-Resolution Using Very Deep Residual Channel Attention Networks](https://mlsanity.com/paper/1807.02758/) (2018)
- [Pre-Trained Image Processing Transformer](https://mlsanity.com/paper/2012.00364/) (2020)
- [Residual Dense Network for Image Super-Resolution](https://mlsanity.com/paper/1802.08797/) (2018)
- [KiloNeRF: Speeding up Neural Radiance Fields with Thousands of Tiny MLPs](https://mlsanity.com/paper/2103.13744/) (2021)
- [Learning Continuous Image Representation with Local Implicit Image Function](https://mlsanity.com/paper/2012.09161/) (2020)
- [Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution](https://mlsanity.com/paper/1704.03915/) (2017)
- [Activating More Pixels in Image Super-Resolution Transformer](https://mlsanity.com/paper/2205.04437/) (2022)
- [NeRV: Neural Representations for Videos](https://mlsanity.com/paper/2110.13903/) (2021)
- [MemNet: A Persistent Memory Network for Image Restoration](https://mlsanity.com/paper/1708.02209/) (2017)
- [Implicit Diffusion Models for Continuous Super-Resolution](https://mlsanity.com/paper/2303.16491/) (2023)
- [Generating Videos with Dynamics-aware Implicit Generative Adversarial Networks](https://mlsanity.com/paper/2202.10571/) (2022)
- [Meta-SR: A Magnification-Arbitrary Network for Super-Resolution](https://mlsanity.com/paper/1903.00875/) (2019)
- [Implicit Neural Representations for Image Compression](https://mlsanity.com/paper/2112.04267/) (2021)
- [Feedback Network for Image Super-Resolution](https://mlsanity.com/paper/1903.09814/) (2019)
- [Modulated Periodic Activations for Generalizable Local Functional Representations](https://mlsanity.com/paper/2104.03960/) (2021)
- [Adversarial Generation of Continuous Images](https://mlsanity.com/paper/2011.12026/) (2020)
- [Local Texture Estimator for Implicit Representation Function](https://mlsanity.com/paper/2111.08918/) (2021)
- [COIN++: Neural Compression Across Modalities](https://mlsanity.com/paper/2201.12904/) (2022)
- [Transformers as Meta-Learners for Implicit Neural Representations](https://mlsanity.com/paper/2208.02801/) (2022)
- [CiaoSR: Continuous Implicit Attention-in-Attention Network for Arbitrary-Scale Image Super-Resolution](https://mlsanity.com/paper/2212.04362/) (2022)
- [Learning A Single Network for Scale-Arbitrary Super-Resolution](https://mlsanity.com/paper/2004.03791/) (2020)
- [UltraSR: Spatial Encoding is a Missing Key for Implicit Image Function-based Arbitrary-Scale Super-Resolution](https://mlsanity.com/paper/2103.12716/) (2021)

## Built on by (most cited first, 21 of 21)

- [Pixel to Gaussian: Ultra-Fast Continuous Super-Resolution with 2D Gaussian Modeling](https://mlsanity.com/paper/2503.06617/) (2025)
- [Generalized and Efficient 2D Gaussian Splatting for Arbitrary-scale Super-Resolution](https://mlsanity.com/paper/2501.06838/) (2025)
- [S3Mamba: Arbitrary-Scale Super-Resolution via Scaleable State Space Model](https://mlsanity.com/paper/2411.11906/) (2024)
- [Gradient as Conditions: Rethinking HOG for All-in-one Image Restoration](https://mlsanity.com/paper/2504.09377/) (2025)
- [Latent Diffusion, Implicit Amplification: Efficient Continuous-Scale Super-Resolution for Remote Sensing Images](https://mlsanity.com/paper/2410.22830/) (2024)
- [IM-LUT: Interpolation Mixing Look-Up Tables for Image Super-Resolution](https://mlsanity.com/paper/2507.09923/) (2025)
- [Scale-Invariant Adversarial Attack against Arbitrary-scale Super-resolution](https://mlsanity.com/paper/2503.04385/) (2025)
- [SR-CACO-2: A Dataset for Confocal Fluorescence Microscopy Image Super-Resolution](https://mlsanity.com/paper/2406.09168/) (2024)
- [LoFi: Neural Local Fields for Scalable Image Reconstruction](https://mlsanity.com/paper/2411.04995/) (2024)
- [Test-time Cost-and-Quality Controllable Arbitrary-Scale Super-Resolution with Variable Fourier Components](https://mlsanity.com/paper/2412.05517/) (2024)
- [Implicit Grid Convolution for Multi-Scale Image Super-Resolution](https://mlsanity.com/paper/2408.09674/) (2024)
- [Towards Robust and Generalizable Continuous Space-Time Video Super-Resolution with Events](https://mlsanity.com/paper/2510.03833/) (2025)
- [OmniScaleSR: Unleashing Scale-Controlled Diffusion Prior for Faithful and Realistic Arbitrary-Scale Image Super-Resolution](https://mlsanity.com/paper/2512.04699/) (2025)
- [Towards Arbitrary-Scale Spacecraft Image Super-Resolution via Salient Region-Guidance](https://mlsanity.com/paper/2504.18127/) (2025)
- [T-MLP: Tailed Multi-Layer Perceptron for Level-of-Detail Signal Representation](https://mlsanity.com/paper/2509.00066/) (2025)
- [Towards Arbitrary Motion Completing via Hierarchical Continuous Representation](https://mlsanity.com/paper/2512.21183/) (2025)
- [GB-LSR: Local Spectral Decoding with a Learned Global Bandwidth for Arbitrary-Scale Super-Resolution](https://mlsanity.com/paper/2606.19617/) (2026)
- [Dual-Prior Guided Null-Space Learning with Mixture-of-Splines for Arbitrary Medical Slice Super-Resolution](https://mlsanity.com/paper/2606.26716/) (2026)
- [Adaptive Densification for High-Fidelity and Efficient Sparse Gaussian Splatting in Arbitrary-Scale Super-Resolution](https://mlsanity.com/paper/2606.29400/) (2026)
- [Resonant Brane Splatting for Arbitrary-Scale Super-Resolution](https://mlsanity.com/paper/2606.29453/) (2026)
- [Super-Resolving Unseen Hyperspectral Sensors at Any Scale via Spatial Operators](https://mlsanity.com/paper/2609.39926/) (2026)

## Abstract (arXiv)

The recent work Local Implicit Image Function (LIIF) and subsequent Implicit Neural Representation (INR) based works have achieved remarkable success in Arbitrary-Scale Super-Resolution (ASSR) by using MLP to decode Low-Resolution (LR) features. However, these continuous image representations typically implement decoding in High-Resolution (HR) High-Dimensional (HD) space, leading to a quadratic increase in computational cost and seriously hindering the practical applications of ASSR. To tackle this problem, we propose a novel Latent Modulated Function (LMF), which decouples the HR-HD decoding process into shared latent decoding in LR-HD space and independent rendering in HR Low-Dimensional (LD) space, thereby realizing the first computational optimal paradigm of continuous image representation. Specifically, LMF utilizes an HD MLP in latent space to generate latent modulations of each LR feature vector. This enables a modulated LD MLP in render space to quickly adapt to any input feature vector and perform rendering at arbitrary resolution. Furthermore, we leverage the positive correlation between modulation intensity and input image complexity to design a Controllable Multi-Scale Rendering (CMSR) algorithm, offering the flexibility to adjust the decoding efficiency based on the rendering precision. Extensive experiments demonstrate that converting existing INR-based ASSR methods to LMF can reduce the computational cost by up to 99.9%, accelerate inference by up to 57 times, and save up to 76% of parameters, while maintaining competitive performance. The code is available at https://github.com/HeZongyao/LMF.