# Non-Uniform Stochastic Average Gradient Method for Training Conditional Random Fields

arXiv: https://arxiv.org/abs/1504.04406 · HTML: https://mlsanity.com/paper/1504.04406/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Apr 2015.

## Built on by (most cited first, 6 of 6)

- [SVRG Meets AdaGrad: Painless Variance Reduction](https://mlsanity.com/paper/2102.09645/) (2021)
- [Stochastic Reweighted Gradient Descent](https://mlsanity.com/paper/2103.12293/) (2021)
- [Structured Convolutional Kernel Networks for Airline Crew Scheduling](https://mlsanity.com/paper/2105.11646/) (2021)
- [Safe Screening for Sparse Conditional Random Fields](https://mlsanity.com/paper/2111.13958/) (2021)
- [Preferential Subsampling for Stochastic Gradient Langevin Dynamics](https://mlsanity.com/paper/2210.16189/) (2022)
- [Stochastic optimization with arbitrary recurrent data sampling](https://mlsanity.com/paper/2401.07694/) (2024)

## Abstract (arXiv)

We apply stochastic average gradient (SAG) algorithms for training conditional random fields (CRFs). We describe a practical implementation that uses structure in the CRF gradient to reduce the memory requirement of this linearly-convergent stochastic gradient method, propose a non-uniform sampling scheme that substantially improves practical performance, and analyze the rate of convergence of the SAGA variant under non-uniform sampling. Our experimental results reveal that our method often significantly outperforms existing methods in terms of the training objective, and performs as well or better than optimally-tuned stochastic gradient methods in terms of test error.