# Fractional is Better: Learnable Derivative Orders in Neural Operator Learning

arXiv: https://arxiv.org/abs/2610.20363 · HTML: https://mlsanity.com/paper/2610.20363/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 48 references that are in the map.
- First posted on arXiv 14 Oct 2026.

## Builds on (most cited first, 25 of 25)

- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators](https://mlsanity.com/paper/1910.03193/) (2019)
- [On the Spectral Bias of Neural Networks](https://mlsanity.com/paper/1806.08734/) (2018)
- [Error bounds for approximations with deep ReLU networks](https://mlsanity.com/paper/1610.01145/) (2016)
- [Physics-Informed Neural Operator for Learning Partial Differential Equations](https://mlsanity.com/paper/2111.03794/) (2021)
- [PDEBENCH: An Extensive Benchmark for Scientific Machine Learning](https://mlsanity.com/paper/2210.07182/) (2022)
- [Choose a Transformer: Fourier or Galerkin](https://mlsanity.com/paper/2105.14995/) (2021)
- [GNOT: A General Neural Operator Transformer for Operator Learning](https://mlsanity.com/paper/2302.14376/) (2023)
- [Transolver: A Fast Transformer Solver for PDEs on General Geometries](https://mlsanity.com/paper/2402.02366/) (2024)
- [Convolutional Neural Operators for robust and accurate learning of PDEs](https://mlsanity.com/paper/2302.01178/) (2023)
- [Transolver++: An Accurate Neural Solver for PDEs on Million-Scale Geometries](https://mlsanity.com/paper/2502.02414/) (2025)
- [Solving High-Dimensional PDEs with Latent Spectral Models](https://mlsanity.com/paper/2301.12664/) (2023)
- [Operator Learning with Neural Fields: Tackling PDEs on General Geometries](https://mlsanity.com/paper/2306.07266/) (2023)
- [Neural Operators with Localized Integral and Differential Kernels](https://mlsanity.com/paper/2402.16845/) (2024)
- [Unleashing the Potential of Fractional Calculus in Graph Neural Networks with FROND](https://mlsanity.com/paper/2404.17099/) (2024)
- [Fourier neural operator for learning solutions to macroscopic traffic flow models: Application to the forward and inverse problems](https://mlsanity.com/paper/2308.07051/) (2023)
- [Enhancing Fourier Neural Operators with Local Spatial Features](https://mlsanity.com/paper/2503.17797/) (2025)
- [Learning-based solutions to nonlinear hyperbolic PDEs: Empirical insights on generalization errors](https://mlsanity.com/paper/2302.08144/) (2023)
- [Neural Variable-Order Fractional Differential Equation Networks](https://mlsanity.com/paper/2503.16207/) (2025)
- [Urban traffic analysis and forecasting through shared Koopman eigenmodes](https://mlsanity.com/paper/2409.04728/) (2024)
- [Generalizing PDE Emulation with Equation-Aware Neural Operators](https://mlsanity.com/paper/2511.09729/) (2025)
- [Exact Stochastic Second Order Deep Learning](https://mlsanity.com/paper/2104.03804/) (2021)
- [Catastrophic Overfitting, Entropy Gap and Participation Ratio: A Noiseless lp Norm Solution for Fast Adversarial Training](https://mlsanity.com/paper/2505.02360/) (2025)
- [FlowMixer: A Depth-Agnostic Neural Architecture for Interpretable Spatiotemporal Forecasting](https://mlsanity.com/paper/2505.16786/) (2025)
- [Efficient Dilated Squeeze and Excitation Neural Operator for Differential Equations](https://mlsanity.com/paper/2601.17407/) (2026)

## Abstract (arXiv)

Neural operators learn mappings between function spaces, enabling fast surrogate solutions to partial differential equations. Despite remarkable architectural diversity, these methods often share a common input representation: raw coordinate-value pairs (x, u(x)). We ask whether inputs aligned with PDE differential structure can improve learning. Through Picard iteration on mild solutions, we show that derivatives of the input appear explicitly in the solution operator, suggesting that providing derivative features should reduce the network's implicit differentiation burden. We prove this intuition: providing derivative features improves approximation bounds from O(W-(s-m)/d) to O(W-s/d), where W is parameter count, s is input regularity, m is the PDE order, and d is spatial dimension. Our central finding, however, is a surprise: the optimal derivative order β^* is strictly less than the PDE order m. This gap arises from a bias-variance tradeoff in spectral space that we characterize in closed form. Learning β from data achieves automatic spectral regularization. We introduce partial-NO (del-NO), for derivative-augmented neural operators, an architecture-agnostic augmentation that provides learnable fractional derivative features to any neural operator backbone. Across benchmark problems and architectures, partial-NO consistently improves prediction accuracy, with learned orders that reflect a representation of known physics modulated by noise and finite-sample constraints.