# On the Powerball Method for Optimization

arXiv: https://arxiv.org/abs/1603.07421 · HTML: https://mlsanity.com/paper/1603.07421/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Mar 2016.

## Built on by (most cited first, 2 of 2)

- [GradPower: Powering Gradients for Faster Language Model Pre-Training](https://mlsanity.com/paper/2505.24275/) (2025)
- [PowerStep: Memory-Efficient Adaptive Optimization via ℓp-Norm Steepest Descent](https://mlsanity.com/paper/2605.10335/) (2026)

## Abstract (arXiv)

We propose a new method to accelerate the convergence of optimization algorithms. This method simply adds a power coefficient γ∈[0,1) to the gradient during optimization. We call this the Powerball method and analyze the convergence rate for the Powerball method for strongly convex functions. While theoretically the Powerball method is guaranteed to have a linear convergence rate in the same order of the gradient method, we show that empirically it significantly outperforms the gradient descent and Newton's method, especially during the initial iterations. We demonstrate that the Powerball method provides a 10-fold speedup of the convergence of both gradient descent and L-BFGS on multiple real datasets.