# Efficient Training of Neural Fractional-Order Differential Equation via Adjoint Backpropagation

arXiv: https://arxiv.org/abs/2503.16666 · HTML: https://mlsanity.com/paper/2503.16666/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 66 references that are in the map.
- First posted on arXiv 20 Mar 2025.

## Builds on (most cited first, 23 of 23)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [GraphSAINT: Graph Sampling Based Inductive Learning Method](https://mlsanity.com/paper/1907.04931/) (2019)
- [Augmented Neural ODEs](https://mlsanity.com/paper/1904.01681/) (2019)
- [Stable Architectures for Deep Neural Networks](https://mlsanity.com/paper/1705.03341/) (2017)
- [GRAND: Graph Neural Diffusion](https://mlsanity.com/paper/2106.10934/) (2021)
- [Graph Neural Ordinary Differential Equations](https://mlsanity.com/paper/1911.07532/) (2019)
- [Continuous Graph Neural Networks](https://mlsanity.com/paper/1912.00967/) (2019)
- [Stable Neural ODE with Lyapunov-Stable Equilibrium Points for Defending Against Adversarial Attacks](https://mlsanity.com/paper/2110.12976/) (2021)
- [Fast Training of Convolutional Networks through FFTs](https://mlsanity.com/paper/1312.5851/) (2013)
- [On Robustness of Neural Ordinary Differential Equations](https://mlsanity.com/paper/1910.05513/) (2019)
- [Anti-Symmetric DGN: a stable architecture for Deep Graph Networks](https://mlsanity.com/paper/2210.09789/) (2022)
- ["Hey, that's not an ODE": Faster ODE Adjoints via Seminorms](https://mlsanity.com/paper/2009.09457/) (2020)
- [RobustLoc: Robust Camera Pose Regression in Challenging Driving Environments](https://mlsanity.com/paper/2211.11238/) (2022)
- [Unleashing the Potential of Fractional Calculus in Graph Neural Networks with FROND](https://mlsanity.com/paper/2404.17099/) (2024)
- [Fractional Deep Neural Network via Constrained Optimization](https://mlsanity.com/paper/2004.00719/) (2020)
- [Coupling Graph Neural Networks with Fractional Order Continuous Dynamics: A Robustness Study](https://mlsanity.com/paper/2401.04331/) (2024)
- [Distributed-Order Fractional Graph Operating Network](https://mlsanity.com/paper/2411.05274/) (2024)

## Built on by (most cited first, 3 of 3)

- [Adaptive Multi-view Graph Contrastive Learning via Fractional-order Neural Diffusion Networks](https://mlsanity.com/paper/2511.06216/) (2025)
- [LongSpike: Fractional Order Spiking State Space Models for Efficient Long Sequence Learning](https://mlsanity.com/paper/2606.12895/) (2026)
- [Fractional State Space Transition for Long Sequence Modeling](https://mlsanity.com/paper/2609.36314/) (2026)

## Abstract (arXiv)

Fractional-order differential equations (FDEs) enhance traditional differential equations by extending the order of differential operators from integers to real numbers, offering greater flexibility in modeling complex dynamical systems with nonlocal characteristics. Recent progress at the intersection of FDEs and deep learning has catalyzed a new wave of innovative models, demonstrating the potential to address challenges such as graph representation learning. However, training neural FDEs has primarily relied on direct differentiation through forward-pass operations in FDE numerical solvers, leading to increased memory usage and computational complexity, particularly in large-scale applications. To address these challenges, we propose a scalable adjoint backpropagation method for training neural FDEs by solving an augmented FDE backward in time, which substantially reduces memory requirements. This approach provides a practical neural FDE toolbox and holds considerable promise for diverse applications. We demonstrate the effectiveness of our method in several tasks, achieving performance comparable to baseline models while significantly reducing computational overhead.