# Accelerated Learning with Robustness to Adversarial Regressors

arXiv: https://arxiv.org/abs/2005.01529 · HTML: https://mlsanity.com/paper/2005.01529/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 May 2020.

## Built on by (most cited first, 3 of 3)

- [Online Algorithms and Policies Using Adaptive and Machine Learning Approaches](https://mlsanity.com/paper/2105.06577/) (2021)
- [A High-order Tuner for Accelerated Learning and Control](https://mlsanity.com/paper/2103.12868/) (2021)
- [On the stability of gradient descent with second order dynamics for time-varying cost functions](https://mlsanity.com/paper/2405.13765/) (2024)

## Abstract (arXiv)

High order momentum-based parameter update algorithms have seen widespread applications in training machine learning models. Recently, connections with variational approaches have led to the derivation of new learning algorithms with accelerated learning guarantees. Such methods however, have only considered the case of static regressors. There is a significant need for parameter update algorithms which can be proven stable in the presence of adversarial time-varying regressors, as is commonplace in control theory. In this paper, we propose a new discrete time algorithm which 1) provides stability and asymptotic convergence guarantees in the presence of adversarial regressors by leveraging insights from adaptive control theory and 2) provides non-asymptotic accelerated learning guarantees leveraging insights from convex optimization. In particular, our algorithm reaches an ε sub-optimal point in at most tildeO(1/√ε) iterations when regressors are constant - matching lower bounds due to Nesterov of Omega(1/√ε), up to a log(1/ε) factor and provides guaranteed bounds for stability when regressors are time-varying. We provide numerical experiments for a variant of Nesterov's provably hard convex optimization problem with time-varying regressors, as well as the problem of recovering an image with a time-varying blur and noise using streaming data.