mlsanityData to 7 Oct 2026

HOIN: High-Order Implicit Neural Representations

arXiv 2404.14674 ↗First posted 23 Apr 2024cs.LG cs.AI cs.CVcited by 3

Not on the map. Why?

Abstract (arXiv)

Implicit neural representations (INR) suffer from worsening spectral bias, which results in overly smooth solutions to the inverse problem. To deal with this problem, we propose a universal framework for processing inverse problems called High-Order Implicit Neural Representations (HOIN). By refining the traditional cascade structure to foster high-order interactions among features, HOIN enhances the model's expressive power and mitigates spectral bias through its neural tangent kernel's (NTK) strong diagonal properties, accelerating and optimizing inverse problem resolution. By analyzing the model's expression space, high-order derivatives, and the NTK matrix, we theoretically validate the feasibility of HOIN. HOIN realizes 1 to 3 dB improvements in most inverse problems, establishing a new state-of-the-art recovery quality and training efficiency, thus providing a new general paradigm for INR and paving the way for it to solve the inverse problem.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 31 of its 44 references that are in the map. First posted on arXiv 23 Apr 2024.

Builds on

31 of 44 references.

References · 31

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  4. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
  5. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,597
  6. 2020Implicit Neural Representations with Periodic Activation Functionscited by 2,214
  7. 2020Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domainscited by 2,170
  8. 2018On the Spectral Bias of Neural Networkscited by 1,321
  9. 2017Deep Image Priorcited by 1,097
  10. 2020Learning Continuous Image Representation with Local Implicit Image Functioncited by 550
  11. 2016The Little Engine that Could: Regularization by Denoising (RED)cited by 303
  12. 2021Beyond Periodicity: Towards a Unifying Framework for Activations in Coordinate-MLPscited by 134
19 more
  1. 2018Deep Decoder: Concise Image Representations from Untrained Non-convolutional Networkscited by 120
  2. 2022HorNet: Efficient High-Order Spatial Interactions with Recursive Gated Convolutionscited by 103
  3. 2023FINER: Flexible spectral-bias tuning in Implicit NEural Representation by Variable-periodic Activation Functionscited by 77
  4. 2019On the Expressive Power of Deep Polynomial Neural Networkscited by 65
  5. 2020Deep Polynomial Neural Networkscited by 63
  6. 2021Quadratic Residual Networks: A New Class of Neural Networks for Solving Forward and Inverse Problems in Physics Involving PDEscited by 42
  7. 2019Denoising and Regularization via Exploiting the Structural Bias of Convolutional Generatorscited by 40
  8. 2023SHACIRA: Scalable HAsh-grid Compression for Implicit Neural Representationscited by 31
  9. 2023INCODE: Implicit Neural Conditioning with Prior Knowledge Embeddingscited by 31
  10. 2022QuadraLib: A Performant Quadratic Neural Network Library for Architecture Optimization and Design Explorationcited by 21
  11. 2019The Spectral Bias of the Deep Image Priorcited by 20
  12. 2021Spectral Analysis of the Neural Tangent Kernel for Deep Residual Networkscited by 20
  13. 2021On Expressivity and Trainability of Quadratic Networkscited by 19
  14. 2022The Spectral Bias of Polynomial Neural Networkscited by 19
  15. 2023Polynomial Implicit Neural Representations For Large Diverse Datasetscited by 18
  16. 2020Neural Architecture Search for Deep Image Priorcited by 14
  17. 2023Regularization of polynomial networks for image recognitioncited by 13
  18. 2017SORT: Second-Order Response Transform for Visual Recognitioncited by 8
  19. 2023Neural Free-Viewpoint Relighting for Glossy Indirect Illuminationcited by 7

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 2.

3 papers.

  1. 2024H-SIREN: Improving implicit neural representations with hyperbolic periodic functionscited by 5
  2. 2025ViSIR: Vision Transformer Single Image Reconstruction Method for Earth System Modelscited by 0
  3. 2025Vis-CoT: A Human-in-the-Loop Framework for Interactive Visualization and Intervention in LLM Chain-of-Thought Reasoningcited by 0