mlsanityData to 14 Oct 2026

Risk and parameter convergence of logistic regression

arXiv 1803.07300 ↗First posted 20 Mar 2018cs.LG math.OC stat.MLcited by 85

Not on the map. Why?

Abstract (arXiv)

Gradient descent, when applied to the task of logistic regression, outputs iterates which are biased to follow a unique ray defined by the data. The direction of this ray is the maximum margin predictor of a maximal linearly separable subset of the data; the gradient descent iterates converge to this ray in direction at the rate O(lnln t / ln t). The ray does not pass through the origin in general, and its offset is the bounded global optimum of the risk over the remaining data; gradient descent recovers this offset at a rate O((ln t)2 / √t).

Key facts. Cited by 85 papers in the map (data through 14 Oct 2026). First posted on arXiv 20 Mar 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 16; 2022: 14; 2023: 18; 2024: 16; 2025: 11; 2026: 10.

85 papers.

Sort:

By area:

  1. 2022On the Implicit Bias in Deep-Learning Algorithmscited by 86
  2. 2021Gradient Descent on Two-layer Nets: Margin Maximization and Simplicity Biascited by 83
  3. 2022Imbalance Trouble: Revisiting Neural-Collapse Geometrycited by 80
  4. 2021Label-Imbalanced and Group-Sensitive Classification under Overparameterizationcited by 74
  5. 2023Transformers as Support Vector Machinescited by 74
  6. 2023Max-Margin Token Selection in Attention Mechanismcited by 58
  7. 2022Implicit Regularization Towards Rank Minimization in ReLU Networkscited by 53
  8. 2025Grokking at the Edge of Numerical Stabilitycited by 46
  9. 2023Implicit Bias of Gradient Descent for Logistic Regression at the Edge of Stabilitycited by 42
  10. 2022Probing Classifiers are Unreliable for Concept Removal and Detectioncited by 41
  11. 2021Understanding the role of importance weighting for deep learningcited by 39
  12. 2022Implicit Bias of Gradient Descent on Reparametrized Models: On Equivalence to Mirror Descentcited by 39
38 more
  1. 2023On the Optimization and Generalization of Multi-head Attentioncited by 37
  2. 2024Large Stepsize Gradient Descent for Logistic Loss: Non-Monotonicity of the Loss Improves Optimization Efficiencycited by 35
  3. 2021Early-stopped neural networks are consistentcited by 34
  4. 2021On Margin Maximization in Linear and ReLU Networkscited by 34
  5. 2024Mechanics of Next Token Prediction with Self-Attentioncited by 34
  6. 2021Fast Margin Maximization via Dual Accelerationcited by 33
  7. 2024Implicit Bias and Fast Convergence Rates for Self-attentioncited by 31
  8. 2023A Fast Optimization View: Reformulating Single Layer Attention in LLM Based on Tensor and SVM Trick, and Solving It in Matrix Multiplication Timecited by 30
  9. 2022Stability vs Implicit Bias of Gradient Methods on Separable Data and Beyondcited by 29
  10. 2023Benign Overfitting in Linear Classifiers and Leaky ReLU Networks from KKT Conditions for Margin Maximizationcited by 29
  11. 2023On the Implicit Bias of Adamcited by 27
  12. 2025Metastable Dynamics of Chain-of-Thought Reasoning: Provable Benefits of Search, RL and Distillationcited by 25
  13. 2024Implicit Geometry of Next-token Prediction: From Language Sparsity Patterns to Model Representationscited by 24
  14. 2023On the Implicit Geometry of Cross-Entropy Parameterizations for Label-Imbalanced Datacited by 23
  15. 2021SGD: The Role of Implicit Regularization, Batch-size and Multiple-epochscited by 22
  16. 2023Generalization and Stability of Interpolating Neural Networks with Minimal Widthcited by 21
  17. 2021Understanding Deflation Process in Over-parametrized Tensor Decompositioncited by 20
  18. 2023Six Lectures on Linearized Neural Networkscited by 20
  19. 2024A Survey on Statistical Theory of Deep Learning: Approximation, Training Dynamics, and Generative Modelscited by 20
  20. 2021Towards Understanding the Data Dependency of Mixup-style Trainingcited by 19
  21. 2021Inductive Bias of Multi-Channel Linear Convolutional Networks with Bounded Weight Normcited by 18
  22. 2021On the Equivalence between Neural Network and Support Vector Machinecited by 17
  23. 2022Stochastic linear optimization never overfits with quadratically-bounded losses on general datacited by 17
  24. 2024Large Stepsize Gradient Descent for Non-Homogeneous Two-Layer Networks: Margin Improvement and Fast Optimizationcited by 17
  25. 2024A Short Survey on Importance Weighting for Machine Learningcited by 15
  26. 2022Feature selection with gradient descent on two-layer networks in low-rotation regimescited by 14
  27. 2024Implicit Optimization Bias of Next-Token Prediction in Linear Modelscited by 14
  28. 2024Transformers Learn Low Sensitivity Functions: Investigations and Implicationscited by 14
  29. 2024Improving Generalization and Convergence by Enhancing Implicit Regularizationcited by 14
  30. 2021Actor-critic is implicitly biased towards high entropy optimal policiescited by 13
  31. 2021The Dynamics of Gradient Descent for Overparametrized Neural Networkscited by 11
  32. 2024Implicit Bias of Mirror Flow on Separable Datacited by 9
  33. 2024Optimizing Attention with Mirror Descent: Generalized Max-Margin Token Selectioncited by 9
  34. 2025Benefits of Early Stopping in Gradient Descent for Overparameterized Logistic Regressioncited by 9
  35. 2025Large Stepsizes Accelerate Gradient Descent for Regularized Logistic Regressioncited by 9
  36. 2022On Generalization of Decentralized Learning with Separable Datacited by 8
  37. 2024Understanding Domain Generalization: A Noise Robustness Perspectivecited by 8
  38. 2023Tight Risk Bounds for Gradient Descent on Separable Datacited by 7

50 most cited of 85 All 85 (JSON)