M"untz-Sz'asz Networks: Neural Architectures with Learnable Power-Law Bases

Standard neural network architectures employ fixed activation functions (ReLU, tanh, sigmoid) that are poorly suited for approximating functions with singular or fractional power behavior, a structure that arises ubiquitously in physics, including boundary layers, fracture mechanics, and corner singularities. We introduce M"untz-Sz'asz Networks (MSN), a novel architecture that replaces fixed smooth activations with learnable fractional power bases grounded in classical approximation theory. Each MSN edge computes φ(x) = sumk ak |x|μk + sumk bk sign(x)|x|λk, where the exponents μk, λk are learned alongside the coefficients. We prove that MSN inherits universal approximation from the M"untz-Sz'asz theorem and establish novel approximation rates: for functions of the form |x|α, MSN achieves error O(|μ - α|2) with a single learned exponent, whereas standard MLPs require O(ε-1/α) neurons for comparable accuracy. On supervised regression with singular target functions, MSN achieves 5-8x lower error than MLPs with 10x fewer parameters. Physics-informed neural networks (PINNs) represent a particularly demanding application for singular function approximation; on PINN benchmarks including a singular ODE and stiff boundary-layer problems, MSN achieves 3-6x improvement while learning interpretable exponents that match the known solution structure. Our results demonstrate that theory-guided architectural design can yield dramatic improvements for scientifically-motivated function classes.