# Convolutional neural networks with fractional order gradient method

arXiv: https://arxiv.org/abs/1905.05336 · HTML: https://mlsanity.com/paper/1905.05336/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 May 2019.

## Built on by (most cited first, 9 of 9)

- [A Caputo fractional derivative-based algorithm for optimization](https://mlsanity.com/paper/2104.02259/) (2021)
- [Performance Analysis of Fractional Learning Algorithms](https://mlsanity.com/paper/2110.05201/) (2021)
- [Fractional-order spike-timing-dependent gradient descent for multi-layer spiking neural networks](https://mlsanity.com/paper/2410.15293/) (2024)
- [Effective Dimension Aware Fractional-Order Stochastic Gradient Descent for Convex Optimization Problems](https://mlsanity.com/paper/2503.13764/) (2025)
- [Using a novel fractional-order gradient method for CNN back-propagation](https://mlsanity.com/paper/2205.00581/) (2022)
- [Convergence Analysis of Fractional Gradient Descent](https://mlsanity.com/paper/2311.18426/) (2023)
- [More Optimal Fractional-Order Stochastic Gradient Descent for Non-Convex Optimization Problems](https://mlsanity.com/paper/2505.02985/) (2025)
- [Fractional-Order Federated Learning](https://mlsanity.com/paper/2602.15380/) (2026)
- [Deep Learning under Fractional-Order Differential Privacy](https://mlsanity.com/paper/2605.09890/) (2026)

## Abstract (arXiv)

This paper proposes a fractional order gradient method for the backward propagation of convolutional neural networks. To overcome the problem that fractional order gradient method cannot converge to real extreme point, a simplified fractional order gradient method is designed based on Caputo's definition. The parameters within layers are updated by the designed gradient method, but the propagations between layers still use integer order gradients, and thus the complicated derivatives of composite functions are avoided and the chain rule will be kept. By connecting every layers in series and adding loss functions, the proposed convolutional neural networks can be trained smoothly according to various tasks. Some practical experiments are carried out in order to demonstrate fast convergence, high accuracy and ability to escape local optimal point at last.