Unveiling the Potential of Superexpressive Networks in Implicit Neural Representations
Not on the map. Why?
Abstract (arXiv)
In this study, we examine the potential of one of the ``superexpressive'' networks in the context of learning neural functions for representing complex signals and performing machine learning downstream tasks. Our focus is on evaluating their performance on computer vision and scientific machine learning tasks including signal representation/inverse problems and solutions of partial differential equations. Through an empirical investigation in various benchmark tasks, we demonstrate that superexpressive networks, as proposed by [Zhang et al. NeurIPS, 2022], which employ a specialized network structure characterized by having an additional dimension, namely width, depth, and ``height'', can surpass recent implicit neural representations that use highly-specialized nonlinear activation functions.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 21 of its 38 references that are in the map. First posted on arXiv 27 Mar 2025.
Builds on
21 of 38 references.
References · 21
- 2017Attention Is All You Needcited by 59,932
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
- 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,590
- 2020Implicit Neural Representations with Periodic Activation Functionscited by 2,214
- 2020Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domainscited by 2,170
- 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
- 2021Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gaugescited by 1,131
- 2023WIRE: Wavelet Implicit Neural Representationscited by 220
- 2021Beyond Periodicity: Towards a Unifying Framework for Activations in Coordinate-MLPscited by 134
9 more
- 2023Separable Physics-Informed Neural Networkscited by 64
- 2020Neural Network Approximation: Three Hidden Layers Are Enoughcited by 63
- 2023Hypernetwork-based Meta-Learning for Low-Rank Physics-Informed Neural Networkscited by 42
- 2021Elementary superexpressive activationscited by 35
- 2024Parameterized Physics-informed Neural Networks for Parameterized PDEscited by 35
- 2016Parametric Exponential Linear Unit for Deep Convolutional Neural Networkscited by 32
- 2024PINNACLE: PINN Adaptive ColLocation and Experimental points selectioncited by 31
- 2023An operator preconditioning perspective on training in physics-informed machine learningcited by 27
- 2022Neural Network Architecture Beyond Width and Depthcited by 17
Built on by
Citing papers per year: 2025: 1.
1 paper.