# Effective Dimension Aware Fractional-Order Stochastic Gradient Descent for Convex Optimization Problems

arXiv: https://arxiv.org/abs/2503.13764 · HTML: https://mlsanity.com/paper/2503.13764/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 26 references that are in the map.
- First posted on arXiv 17 Mar 2025.

## Builds on (most cited first, 7 of 7)

- [Understanding deep learning requires rethinking generalization](https://mlsanity.com/paper/1611.03530/) (2016)
- [Optimizing Neural Networks with Kronecker-factored Approximate Curvature](https://mlsanity.com/paper/1503.05671/) (2015)
- [Fisher-Rao Metric, Geometry, and Complexity of Neural Networks](https://mlsanity.com/paper/1711.01530/) (2017)
- [Universal Statistics of Fisher Information in Deep Neural Networks: Mean Field Approach](https://mlsanity.com/paper/1806.01316/) (2018)
- [A scale-dependent notion of effective dimension](https://mlsanity.com/paper/2001.10872/) (2020)
- [Convolutional neural networks with fractional order gradient method](https://mlsanity.com/paper/1905.05336/) (2019)
- [A Two-Scale Complexity Measure for Deep Learning Models](https://mlsanity.com/paper/2401.09184/) (2024)

## Built on by (most cited first, 1 of 1)

- [Fractional Optimizers Meet Fractal Activation Functions: An Empirical Study of Multi-Scale Optimization in Neural Network](https://mlsanity.com/paper/2608.14636/) (2026)

## Abstract (arXiv)

Fractional-order stochastic gradient descent (FOSGD) leverages fractional exponents to capture long-memory effects in optimization. However, its utility is often limited by the difficulty of tuning and stabilizing these exponents. We propose 2SED Fractional-Order Stochastic Gradient Descent (2SEDFOSGD), which integrates the Two-Scale Effective Dimension (2SED) algorithm with FOSGD to adapt the fractional exponent in a data-driven manner. By tracking model sensitivity and effective dimensionality, 2SEDFOSGD dynamically modulates the exponent to mitigate oscillations and hasten convergence. Theoretically, this approach preserves the advantages of fractional memory without the sluggish or unstable behavior observed in na"ive fractional SGD. Empirical evaluations in Gaussian and α-stable noise scenarios using an autoregressive (AR) modeltextcolorred, as well as on the MNIST and CIFAR-100 datasets for image classification, highlight faster convergence and more robust parameter estimates compared to baseline methods, underscoring the potential of dimension-aware fractional techniques for advanced modeling and estimation tasks.