mlsanityData to 14 Oct 2026

Gradient Descent on Logistic Regression: Do Large Step-Sizes Work with Data on the Sphere?

arXiv 2507.11228 ↗First posted 15 Jul 2025cs.LG math.OCcited by 5

Not on the map. Why?

Abstract (arXiv)

Gradient descent (GD) on logistic regression has many fascinating properties. When the dataset is linearly separable, it is known that the iterates converge in direction to the maximum-margin separator regardless of how large the step size is. In the non-separable case, however, it has been shown that GD can exhibit a cycling behaviour even when the step sizes is still below the stability threshold 2/λ, where λ is the largest eigenvalue of the Hessian at the solution. This short paper explores whether restricting the data to have equal magnitude is a sufficient condition for global convergence, under any step size below the stability threshold. We prove that this is true in a one dimensional space, but in higher dimensions cycling behaviour can still occur. We hope to inspire further studies on quantifying how common these cycles are in realistic datasets, as well as finding sufficient conditions to guarantee global convergence with large step sizes.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 4 of its 5 references that are in the map. First posted on arXiv 15 Jul 2025.

Builds on

4 of 5 references.

References · 4

  1. 2017The Implicit Bias of Gradient Descent on Separable Datacited by 655
  2. 2023Implicit Bias of Gradient Descent for Logistic Regression at the Edge of Stabilitycited by 42
  3. 2024Large Stepsize Gradient Descent for Logistic Loss: Non-Monotonicity of the Loss Improves Optimization Efficiencycited by 35
  4. 2024Gradient Descent on Logistic Regression with Non-Separable Data and Large Step Sizescited by 14

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 5.

5 papers.

  1. 2026Tight Bounds for Logistic Regression with Large Stepsize Gradient Descent in Low Dimensioncited by 1
  2. 2026Finite-Sample Performance of Gradient Descent in Logistic Regression with Gaussian Designcited by 1
  3. 2026Tight Transition Time Bounds for Separable Logistic Regression at the Edge of Stabilitycited by 0
  4. 2026Large Stepsizes Federated Learning on Logistic Regression with Linearly Separable Data: The Case of Heterogeneous Devicescited by 0
  5. 2026Improved Convergence of Large Stepsize Gradient Descent for Logistic Regressioncited by 0