mlsanityData to 7 Oct 2026

Margins, Shrinkage, and Boosting

arXiv 1303.4172 ↗First posted 18 Mar 2013cs.LG stat.MLcited by 35

Not on the map. Why?

Abstract (arXiv)

This manuscript shows that AdaBoost and its immediate variants can produce approximate maximum margin classifiers simply by scaling step size choices with a fixed small constant. In this way, when the unscaled step size is an optimal choice, these results provide guarantees for Friedman's empirically successful "shrinkage" procedure for gradient boosting (Friedman, 2000). Guarantees are also provided for a variety of other step sizes, affirming the intuition that increasingly regularized line searches provide improved margin guarantees. The results hold for the exponential loss and similar losses, most notably the logistic loss.

Key facts. Cited by 35 papers in the map (data through 7 Oct 2026). First posted on arXiv 18 Mar 2013.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 8; 2022: 7; 2023: 7; 2024: 6; 2025: 4; 2026: 3.

35 papers.

Sort:

  1. 2023Max-Margin Token Selection in Attention Mechanismcited by 57
  2. 2023On the Optimization and Generalization of Multi-head Attentioncited by 37
  3. 2023(S)GD over Diagonal Linear Networks: Implicit Regularisation, Large Stepsizes and Edge of Stabilitycited by 35
  4. 2024Large Stepsize Gradient Descent for Logistic Loss: Non-Monotonicity of the Loss Improves Optimization Efficiencycited by 34
  5. 2021Fast Margin Maximization via Dual Accelerationcited by 33
  6. 2025Implicit Bias of Spectral Descent and Muon on Multiclass Separable Datacited by 33
  7. 2024The Implicit Bias of Adam on Separable Datacited by 32
  8. 2022Mirror Descent Maximizes Generalized Margin and Can Be Implemented Efficientlycited by 25
  9. 2021Does Momentum Change the Implicit Regularization on Separable Data?cited by 21
  10. 2022Stochastic linear optimization never overfits with quadratically-bounded losses on general datacited by 16
  11. 2021Actor-critic is implicitly biased towards high entropy optimal policiescited by 13
  12. 2022Feature selection with gradient descent on two-layer networks in low-rotation regimescited by 13
23 more
  1. 2023A Unified Approach to Controlling Implicit Regularization via Mirror Descentcited by 13
  2. 2024Flavors of Margin: Implicit Bias of Steepest Descent in Homogeneous Neural Networkscited by 13
  3. 2022Why adversarial training can hurt robust accuracycited by 12
  4. 2025Benefits of Early Stopping in Gradient Descent for Overparameterized Logistic Regressioncited by 9
  5. 2022Thinking Outside the Ball: Optimal Learning with Gradient Descent for Generalized Linear Stochastic Convex Optimizationcited by 7
  6. 2023Gradient Descent Converges Linearly for Logistic Regression on Separable Datacited by 7
  7. 2024The Surprising Harmfulness of Benign Overfitting for Adversarial Robustnesscited by 7
  8. 2024On the Benefits of Over-parameterization for Out-of-Distribution Generalizationcited by 7
  9. 2021Implicit Regularization of Bregman Proximal Point Algorithm and Mirror Descent on Separable Datacited by 6
  10. 2021Bridging the Gap Between Adversarial Robustness and Optimization Biascited by 5
  11. 2021On the Self-Penalization Phenomenon in Feature Selectioncited by 4
  12. 2024The Price of Implicit Bias in Adversarially Robust Generalizationcited by 4
  13. 2023Fast Convergence in Learning Two-Layer Neural Networks with Separable Datacited by 3
  14. 2021AdaBoost and robust one-bit compressed sensingcited by 2
  15. 2021The perils of being unhinged: On the accuracy of classifiers minimizing a noise-robust convex losscited by 2
  16. 2025Basic Inequalities for First-Order Optimization with Applications to Statistical Risk Analysiscited by 2
  17. 2022Blessings and Curses of Covariate Shifts: Adversarial Learning Dynamics, Directional Convergence, and Equilibriacited by 1
  18. 2022Tight bounds for maximum ℓ1-margin classifierscited by 0
  19. 2023Faster Margin Maximization Rates for Generic and Adversarially Robust Optimization Methodscited by 0
  20. 2025Uniform convergence of the smooth calibration error and its relationship with functional gradientcited by 0
  21. 2026When Does ℓ2-Boosting Overfit Benignly? High-Dimensional Risk Asymptotics and the ℓ1 Implicit Biascited by 0
  22. 2026Trajectory-Based Difficulty Scoring for Reliable Learning on Tabular Datacited by 0
  23. 2026Stability Annealing Selects the Implicit Bias of Smoothed Sign Descent: A Rate-Indexed Barrier Path on Separable Datacited by 0