mlsanityData to 7 Oct 2026

Rational neural networks

arXiv 2004.01902 ↗First posted 4 Apr 2020cs.NE cs.LG cs.NAcited by 75

Not on the map. Why?

Abstract (arXiv)

We consider neural networks with rational activation functions. The choice of the nonlinear activation function in deep learning architectures is crucial and heavily impacts the performance of a neural network. We establish optimal bounds in terms of network complexity and prove that rational neural networks approximate smooth functions more efficiently than ReLU networks with exponentially smaller depth. The flexibility and smoothness of rational activation functions make them an attractive alternative to ReLU, as we demonstrate with numerical experiments.

Key facts. Cited by 75 papers in the map (data through 7 Oct 2026). First posted on arXiv 4 Apr 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 10; 2022: 6; 2023: 12; 2024: 17; 2025: 13; 2026: 17.

75 papers.

Sort:

By area:

  1. 2023Convolutional Neural Operators for robust and accurate learning of PDEscited by 207
  2. 2021Activation Functions in Deep Learning: A Comprehensive Survey and Benchmarkcited by 153
  3. 2023A Mathematical Guide to Operator Learningcited by 74
  4. 2024Kolmogorov-Arnold Transformercited by 56
  5. 2021Can stable and accurate neural networks be computed? -- On the barriers of deep learning and Smale's 18th problemcited by 53
  6. 2021Data-driven discovery of Green's functions with human-understandable deep learningcited by 43
  7. 2024rKAN: Rational Kolmogorov-Arnold Networkscited by 31
  8. 2021Learning elliptic partial differential equations with randomized linear algebracited by 29
  9. 2022Learning Green's functions associated with time-dependent partial differential equationscited by 25
  10. 2021PDE-READ: Human-readable Partial Differential Equation Discovery using Deep Learningcited by 24
  11. 2021Adaptive Rational Activations to Boost Deep Reinforcement Learningcited by 23
  12. 2025From Zero to Detail: Deconstructing Ultra-High-Definition Image Restoration from Progressive Spectral Perspectivecited by 22
38 more
  1. 2024Geometry of Polynomial Neural Networkscited by 21
  2. 2025Algebra Unveils Deep Learning -- An Invitation to Neuroalgebraic Geometrycited by 21
  3. 2022PDE-LEARN: Using Deep Learning to Discover Partial Differential Equations from Noisy, Limited Datacited by 20
  4. 2021Reproducing Activation Function for Deep Learningcited by 19
  5. 2021Bridging the Gap between Spatial and Spectral Domains: A Unified Framework for Graph Neural Networkscited by 18
  6. 2023Randomized Sparse Neural Galerkin Schemes for Solving Evolution Equations with Deep Networkscited by 18
  7. 2022Discovery of partial differential equations from highly noisy and sparse data with physics-informed information criterioncited by 16
  8. 2023Weak-PDE-LEARN: A Weak Form Based Approach to Discovering PDEs From Noisy, Limited Datacited by 16
  9. 2024Activation degree thresholds and expressiveness of polynomial neural networkscited by 14
  10. 2022Adaptable Adapterscited by 13
  11. 2023Learning to Optimize for Reinforcement Learningcited by 12
  12. 2024On the Geometry and Optimization of Polynomial Convolutional Networkscited by 11
  13. 2022On Fast Simulation of Dynamical System with Neural Vector Enhanced Numerical Solvercited by 9
  14. 2024Polynomial Composition Activations: Unleashing the Dynamics of Large Language Modelscited by 9
  15. 2025Learning on a Razor's Edge: Identifiability and Singularity of Polynomial Neural Networkscited by 9
  16. 2024Operator learning without the adjointcited by 8
  17. 2024PEDRO: Parameter-Efficient Fine-tuning with Prompt DEpenDent Representation MOdificationcited by 8
  18. 2023Covering Number of Real Algebraic Varieties and Beyond: Improved Bounds and Applicationscited by 7
  19. 2024Learning smooth functions in high dimensions: from sparse polynomials to deep neural networkscited by 7
  20. 2021A generalization of the randomized singular value decompositioncited by 6
  21. 2025Neural Green's Functionscited by 6
  22. 2022SignReLU neural network and its approximation abilitycited by 4
  23. 2023AttNS: Attention-Inspired Numerical Solving For Limited Data Scenarioscited by 4
  24. 2024PETScML: Second-order solvers for training regression problems in Scientific Machine Learningcited by 4
  25. 2024Cauchy activation function and XNetcited by 4
  26. 2024Green Multigrid Networkcited by 3
  27. 2024An explainable operator approximation framework under the guideline of Green's functioncited by 3
  28. 2025Orthogonal greedy algorithm for linear operator learning with shallow neural networkcited by 3
  29. 2025KAN or MLP? Point Cloud Shows the Way Forwardcited by 3
  30. 2025Learning Singularity-Encoded Green's Functions with Application to Iterative Methodscited by 3
  31. 2026Data-driven discovery of governing differential equations across physical systemscited by 3
  32. 2024Convergence Analysis for Deep Sparse Coding via Convolutional Neural Networkscited by 2
  33. 2024Graph Neural Networks Need Cluster-Normalize-Activate Modulescited by 2
  34. 2025C1-approximation with rational functions and rational neural networkscited by 2
  35. 2026Rational ANOVA Networkscited by 2
  36. 2024IAPT: Instruction-Aware Prompt Tuning for Large Language Modelscited by 1
  37. 2025chebgreen: Learning and Interpolating Continuous Empirical Green's Functions from Datacited by 1
  38. 2026From Fuzzy to Exact: The Halo Architecture for Infinite-Depth Reasoning via Rational Arithmeticcited by 1

50 most cited of 75 All 75 (JSON)