# AURA: Angular Update Rate Adaptation for training complex-valued neural networks

arXiv: https://arxiv.org/abs/2609.25914 · HTML: https://mlsanity.com/paper/2609.25914/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 62 references that are in the map.
- First posted on arXiv 22 Sep 2026.

## Builds on (most cited first, 20 of 20)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Optimization Methods for Large-Scale Machine Learning](https://mlsanity.com/paper/1606.04838/) (2016)
- [On the Convergence of Adam and Beyond](https://mlsanity.com/paper/1904.09237/) (2019)
- [AdaBelief Optimizer: Adapting Stepsizes by the Belief in Observed Gradients](https://mlsanity.com/paper/2010.07468/) (2020)
- [Deep Complex Networks](https://mlsanity.com/paper/1705.09792/) (2017)
- [Dissecting Adam: The Sign, Magnitude and Variance of Stochastic Gradients](https://mlsanity.com/paper/1705.07774/) (2017)
- [Descending through a Crowded Valley - Benchmarking Deep Learning Optimizers](https://mlsanity.com/paper/2007.01547/) (2020)
- [Online Learning Rate Adaptation with Hypergradient Descent](https://mlsanity.com/paper/1703.04782/) (2017)
- [On Empirical Comparisons of Optimizers for Deep Learning](https://mlsanity.com/paper/1910.05446/) (2019)
- [Weak baselines and reporting biases lead to overoptimism in machine learning for fluid-related partial differential equations](https://mlsanity.com/paper/2407.07218/) (2024)
- [Optimizing the Optimizer for Physics-Informed Neural Networks and Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2501.16371/) (2025)
- [DeepOBS: A Deep Learning Optimizer Benchmark Suite](https://mlsanity.com/paper/1903.05499/) (2019)
- [diffGrad: An Optimization Method for Convolutional Neural Networks](https://mlsanity.com/paper/1909.11015/) (2019)
- [OptunaHub: A Platform for Black-Box Optimization](https://mlsanity.com/paper/2510.02798/) (2025)
- [AngularGrad: A New Optimization Technique for Angular Convergence of Convolutional Neural Networks](https://mlsanity.com/paper/2105.10190/) (2021)
- [Physics-Informed Holomorphic Neural Networks (PIHNNs): Solving Linear Elasticity Problems](https://mlsanity.com/paper/2407.01088/) (2024)
- [Lifelong Machine Learning Potentials](https://mlsanity.com/paper/2303.05911/) (2023)
- [Torque-Aware Momentum](https://mlsanity.com/paper/2412.18790/) (2024)
- [A holomorphic neural network framework for 3D boundary value problems governed by harmonic potentials](https://mlsanity.com/paper/2605.31231/) (2026)
- [Adaptive Gradient Method with Resilience and Momentum](https://mlsanity.com/paper/2010.11041/) (2020)

## Abstract (arXiv)

Complex-valued neural networks (CVNNs) are increasingly adopted for complex-valued data; however, they are often trained with first-order optimizers inherited from the real-valued case. The efficiency of these methods depends largely on the step size, and their step-size rules ignore the angular information available in the complex plane. We address step-size adaptation in the complex domain by introducing AURA (Angular Update Rate Adaptation), a per-parameter step-size adaptation that can be added on top of any first-order optimizer, and removed from it, without altering its update direction. AURA measures the agreement between consecutive updates of each complex parameter, in length, alignment, and sense of rotation, and enlarges the step when they are consistent and reduces it when they are not. It requires no additional gradient evaluations and only inexpensive vector operations per step. We combine AURA with Adam and Muon and compare the resulting methods with well-known first-order optimizers on four test cases of increasing complexity, ranging from the approximation of scalar complex functions to physics-informed training. Fully connected neural networks are used throughout this work. All hyperparameters other than the step size are held fixed across test cases; for one case, we also tune the hyperparameters of each optimizer under the same budget. Our empirical tests show that AURA improves the convergence of its base optimizer in most cases with a small per-step overhead, and we identify the conditions under which it fails to do so.