# What Objective Does Self-paced Learning Indeed Optimize?

arXiv: https://arxiv.org/abs/1511.06049 · HTML: https://mlsanity.com/paper/1511.06049/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Nov 2015.

## Built on by (most cited first, 8 of 8)

- [Few-Cost Salient Object Detection with Adversarial-Paced Learning](https://mlsanity.com/paper/2104.01928/) (2021)
- [A Too-Good-to-be-True Prior to Reduce Shortcut Reliance](https://mlsanity.com/paper/2102.06406/) (2021)
- [Incompatibility Clustering as a Defense Against Backdoor Poisoning Attacks](https://mlsanity.com/paper/2105.03692/) (2021)
- [Unsupervised feature selection via self-paced learning and low-redundant regularization](https://mlsanity.com/paper/2112.07227/) (2021)
- [MAPS: A Noise-Robust Progressive Learning Approach for Source-Free Domain Adaptive Keypoint Detection](https://mlsanity.com/paper/2302.04589/) (2023)
- [Learning to Reason with Curriculum I: Provable Benefits of Autocurriculum](https://mlsanity.com/paper/2603.18325/) (2026)
- [Self-paced Principal Component Analysis](https://mlsanity.com/paper/2106.13880/) (2021)
- [GAGA: Deciphering Age-path of Generalized Self-paced Regularizer](https://mlsanity.com/paper/2209.07063/) (2022)

## Abstract (arXiv)

Self-paced learning (SPL) is a recently raised methodology designed through simulating the learning principle of humans/animals. A variety of SPL realization schemes have been designed for different computer vision and pattern recognition tasks, and empirically substantiated to be effective in these applications. However, the investigation on its theoretical insight is still a blank. To this issue, this study attempts to provide some new theoretical understanding under the SPL scheme. Specifically, we prove that the solving strategy on SPL accords with a majorization minimization algorithm implemented on a latent objective function. Furthermore, we find that the loss function contained in this latent objective has a similar configuration with non-convex regularized penalty (NSPR) known in statistics and machine learning. Such connection inspires us discovering more intrinsic relationship between SPL regimes and NSPR forms, like SCAD, LOG and EXP. The robustness insight under SPL can then be finely explained. We also analyze the capability of SPL on its easy loss prior embedding property, and provide an insightful interpretation to the effectiveness mechanism under previous SPL variations. Besides, we design a group-partial-order loss prior, which is especially useful to weakly labeled large-scale data processing tasks. Through applying SPL with this loss prior to the FCVID dataset, which is currently one of the biggest manually annotated video dataset, our method achieves state-of-the-art performance beyond previous methods, which further helps supports the proposed theoretical arguments.