# Asynchronous Stochastic Quasi-Newton MCMC for Non-Convex Optimization

arXiv: https://arxiv.org/abs/1806.02617 · HTML: https://mlsanity.com/paper/1806.02617/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Jun 2018.

## Built on by (most cited first, 4 of 4)

- [Distributed Learning Systems with First-order Methods](https://mlsanity.com/paper/2104.05245/) (2021)
- [Persia: An Open, Hybrid System Scaling Deep Learning-based Recommenders up to 100 Trillion Parameters](https://mlsanity.com/paper/2111.05897/) (2021)
- [BAGUA: Scaling up Distributed Learning with System Relaxations](https://mlsanity.com/paper/2107.01499/) (2021)
- [Solving Linear Inverse Problems using Higher-Order Annealed Langevin Diffusion](https://mlsanity.com/paper/2305.05014/) (2023)

## Abstract (arXiv)

Recent studies have illustrated that stochastic gradient Markov Chain Monte Carlo techniques have a strong potential in non-convex optimization, where local and global convergence guarantees can be shown under certain conditions. By building up on this recent theory, in this study, we develop an asynchronous-parallel stochastic L-BFGS algorithm for non-convex optimization. The proposed algorithm is suitable for both distributed and shared-memory settings. We provide formal theoretical analysis and show that the proposed method achieves an ergodic convergence rate of cal O(1/√N) (N being the total number of iterations) and it can achieve a linear speedup under certain conditions. We perform several experiments on both synthetic and real datasets. The results support our theory and show that the proposed algorithm provides a significant speedup over the recently proposed synchronous distributed L-BFGS algorithm.