mlsanityData to 7 Oct 2026

Neural networks and rational functions

arXiv 1706.03301 ↗First posted 11 Jun 2017cs.LG cs.NE stat.MLcited by 40

Not on the map. Why?

Abstract (arXiv)

Neural networks and rational functions efficiently approximate each other. In more detail, it is shown here that for any ReLU network, there exists a rational function of degree O(polylog(1/ε)) which is ε-close, and similarly for any rational function there exists a ReLU network of size O(polylog(1/ε)) which is ε-close. By contrast, polynomials need degree Omega(poly(1/ε)) to approximate even a single ReLU. When converting a ReLU network to a rational function as above, the hidden constants depend exponentially on the number of layers, which is shown to be tight; in other words, a compositional representation can be beneficial even for rational functions.

Key facts. Cited by 40 papers in the map (data through 7 Oct 2026). First posted on arXiv 11 Jun 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 8; 2022: 4; 2023: 5; 2024: 8; 2025: 5; 2026: 10.

40 papers.

Sort:

By area:

  1. 2023Convolutional Neural Operators for robust and accurate learning of PDEscited by 207
  2. 2021Activation Functions in Deep Learning: A Comprehensive Survey and Benchmarkcited by 153
  3. 2024Kolmogorov-Arnold Transformercited by 56
  4. 2021Can stable and accurate neural networks be computed? -- On the barriers of deep learning and Smale's 18th problemcited by 53
  5. 2024rKAN: Rational Kolmogorov-Arnold Networkscited by 31
  6. 2021Consistent Sparse Deep Learning: Theory and Computationcited by 27
  7. 2021Adaptive Rational Activations to Boost Deep Reinforcement Learningcited by 23
  8. 2022Why Robust Generalization in Deep Learning is Difficult: Perspective of Expressive Powercited by 23
  9. 2021Neural tensor contractions and the expressive power of deep neural quantum statescited by 21
  10. 2021Parametric Complexity Bounds for Approximating PDEs with Neural Networkscited by 18
  11. 2021Bridging the Gap between Spatial and Spectral Domains: A Unified Framework for Graph Neural Networkscited by 18
  12. 2023Learning to Optimize for Reinforcement Learningcited by 12
28 more
  1. 2024On the Geometry and Optimization of Polynomial Convolutional Networkscited by 11
  2. 2024Polynomial Composition Activations: Unleashing the Dynamics of Large Language Modelscited by 9
  3. 2023Covering Number of Real Algebraic Varieties and Beyond: Improved Bounds and Applicationscited by 7
  4. 2024Learning smooth functions in high dimensions: from sparse polynomials to deep neural networkscited by 7
  5. 2024Statistical guarantees for denoising reflected diffusion modelscited by 6
  6. 2022On Regularizing Coordinate-MLPscited by 5
  7. 2022Transformers with Learnable Activation Functionscited by 5
  8. 2022SignReLU neural network and its approximation abilitycited by 4
  9. 2024Cauchy activation function and XNetcited by 4
  10. 2025Don't Forget the Nonlinearity: Unlocking Activation Functions in Efficient Fine-Tuningcited by 4
  11. 2025Establishing a Scale for Kullback-Leibler Divergence in Language Models Across Various Settingscited by 3
  12. 2024Graph Neural Networks Need Cluster-Normalize-Activate Modulescited by 2
  13. 2025C1-approximation with rational functions and rational neural networkscited by 2
  14. 2026Rational ANOVA Networkscited by 2
  15. 2021STENCIL-NET: Data-driven solution-adaptive discretization of partial differential equationscited by 1
  16. 2025PO-CKAN:Physics Informed Deep Operator Kolmogorov Arnold Networks with Chunk Rational Structurecited by 1
  17. 2026Rational Neural Networks have Expressivity Advantagescited by 1
  18. 2026Quantitative Approximation Rates for Group Equivariant Learningcited by 1
  19. 2026A Universal Reproducing Kernel Hilbert Space from Polynomial Alignment and IMQ Distancecited by 1
  20. 2023A comparison of rational and neural network based approximationscited by 0
  21. 2023Rational Neural Network Controllerscited by 0
  22. 2025On the algorithmic construction of deep ReLU networkscited by 0
  23. 2026Symbolic recovery of PDEs from measurement datacited by 0
  24. 2026CFNN: Continued Fraction Neural Networkcited by 0
  25. 2026Bifurcation Models: Learning Set-Valued Solution Maps with Weight-Tied Dynamicscited by 0
  26. 2026Lower bounds for one-layer transformers that compute paritycited by 0
  27. 2026Rational Sparse Autoencodercited by 0
  28. 2026Deep Q-Learning on H"older Spacescited by 0