# GAGA: Deciphering Age-path of Generalized Self-paced Regularizer

arXiv: https://arxiv.org/abs/2209.07063 · HTML: https://mlsanity.com/paper/2209.07063/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 58 references that are in the map.
- First posted on arXiv 15 Sep 2022.

## Builds on (most cited first, 8 of 8)

- [OpenML: networked science in machine learning](https://mlsanity.com/paper/1407.7722/) (2014)
- [Robust Loss Functions under Label Noise for Deep Neural Networks](https://mlsanity.com/paper/1712.09482/) (2017)
- [Curriculum Learning: A Survey](https://mlsanity.com/paper/2101.10382/) (2021)
- [Manipulating Machine Learning: Poisoning Attacks and Countermeasures for Regression Learning](https://mlsanity.com/paper/1804.00308/) (2018)
- [Active Self-Paced Learning for Cost-Effective and Progressive Face Identification](https://mlsanity.com/paper/1701.03555/) (2017)
- [Self-Paced Deep Regression Forests with Consideration on Underrepresented Examples](https://mlsanity.com/paper/2004.01459/) (2020)
- [What Objective Does Self-paced Learning Indeed Optimize?](https://mlsanity.com/paper/1511.06049/) (2015)
- [On the structure of regularization paths for piecewise differentiable regularization terms](https://mlsanity.com/paper/2111.06775/) (2021)

## Abstract (arXiv)

Nowadays self-paced learning (SPL) is an important machine learning paradigm that mimics the cognitive process of humans and animals. The SPL regime involves a self-paced regularizer and a gradually increasing age parameter, which plays a key role in SPL but where to optimally terminate this process is still non-trivial to determine. A natural idea is to compute the solution path w.r.t. age parameter (i.e., age-path). However, current age-path algorithms are either limited to the simplest regularizer, or lack solid theoretical understanding as well as computational efficiency. To address this challenge, we propose a novel underlineGeneralized underlineAge-path underlineAlgorithm (GAGA) for SPL with various self-paced regularizers based on ordinary differential equations (ODEs) and sets control, which can learn the entire solution spectrum w.r.t. a range of age parameters. To the best of our knowledge, GAGA is the first exact path-following algorithm tackling the age-path for general self-paced regularizer. Finally the algorithmic steps of classic SVM and Lasso are described in detail. We demonstrate the performance of GAGA on real-world datasets, and find considerable speedup between our algorithm and competing baselines.