# Self-Paced Learning: an Implicit Regularization Perspective

arXiv: https://arxiv.org/abs/1606.00128 · HTML: https://mlsanity.com/paper/1606.00128/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Jun 2016.

## Built on by (most cited first, 21 of 21)

- [Curriculum Learning: A Survey](https://mlsanity.com/paper/2101.10382/) (2021)
- [Robust Physical-World Attacks on Face Recognition](https://mlsanity.com/paper/2109.09320/) (2021)
- [Deep Anomaly Discovery From Unlabeled Videos via Normality Advantage and Self-Paced Refinement](https://mlsanity.com/paper/2108.01975/) (2021)
- [Unsupervised Face Morphing Attack Detection via Self-paced Anomaly Detection](https://mlsanity.com/paper/2208.05787/) (2022)
- [Which Samples Should be Learned First: Easy or Hard?](https://mlsanity.com/paper/2110.05481/) (2021)
- [A Curriculum-style Self-training Approach for Source-Free Semantic Segmentation](https://mlsanity.com/paper/2106.11653/) (2021)
- [Learning Rate Curriculum](https://mlsanity.com/paper/2205.09180/) (2022)
- [REALM: Robust Entropy Adaptive Loss Minimization for Improved Single-Sample Test-Time Adaptation](https://mlsanity.com/paper/2309.03964/) (2023)
- [Stochastic Re-weighted Gradient Descent via Distributionally Robust Optimization](https://mlsanity.com/paper/2306.09222/) (2023)
- [Data Optimization in Deep Learning: A Survey](https://mlsanity.com/paper/2310.16499/) (2023)
- [Robust Positive-Unlabeled Learning via Noise Negative Sample Self-correction](https://mlsanity.com/paper/2308.00279/) (2023)
- [Learning Deep Neural Networks under Agnostic Corrupted Supervision](https://mlsanity.com/paper/2102.06735/) (2021)
- [CBM: Curriculum by Masking](https://mlsanity.com/paper/2407.05193/) (2024)
- [Is Data Valuation Learnable and Interpretable?](https://mlsanity.com/paper/2406.02612/) (2024)
- [Self-Paced Learning Strategy with Easy Sample Prior Based on Confidence for the Flying Bird Object Detection Model Training](https://mlsanity.com/paper/2412.06306/) (2024)
- [Adaptively Point-weighting Curriculum Learning](https://mlsanity.com/paper/2505.01665/) (2025)
- [Training With Data Dependent Dynamic Learning Rates](https://mlsanity.com/paper/2105.13464/) (2021)
- [Differentiated Relevances Embedding for Group-based Referring Expression Comprehension](https://mlsanity.com/paper/2203.06382/) (2022)
- [UniFed: A Universal Federation of a Mixture of Highly Heterogeneous Medical Image Classification Tasks](https://mlsanity.com/paper/2408.07075/) (2024)
- [Co-Paced Learning Strategy Based on Confidence for Flying Bird Object Detection Model Training](https://mlsanity.com/paper/2501.12071/) (2025)
- [Prototype-Guided Curriculum Learning for Zero-Shot Learning](https://mlsanity.com/paper/2508.07771/) (2025)

## Abstract (arXiv)

Self-paced learning (SPL) mimics the cognitive mechanism of humans and animals that gradually learns from easy to hard samples. One key issue in SPL is to obtain better weighting strategy that is determined by minimizer function. Existing methods usually pursue this by artificially designing the explicit form of SPL regularizer. In this paper, we focus on the minimizer function, and study a group of new regularizer, named self-paced implicit regularizer that is deduced from robust loss function. Based on the convex conjugacy theory, the minimizer function for self-paced implicit regularizer can be directly learned from the latent loss function, while the analytic form of the regularizer can be even known. A general framework (named SPL-IR) for SPL is developed accordingly. We demonstrate that the learning procedure of SPL-IR is associated with latent robust loss functions, thus can provide some theoretical inspirations for its working mechanism. We further analyze the relation between SPL-IR and half-quadratic optimization. Finally, we implement SPL-IR to both supervised and unsupervised tasks, and experimental results corroborate our ideas and demonstrate the correctness and effectiveness of implicit regularizers.