{"total":74,"rows":[{"id":"2105.12221","title":"Geometry of the Loss Landscape in Overparameterized Neural Networks: Symmetries and Invariances","created":"2021-05-25","cited_by_count":127,"area":null},{"id":"2106.15933","title":"Saddle-to-Saddle Dynamics in Deep Linear Networks: Small Initialization Training, Symmetry, and Sparsity","created":"2021-06-30","cited_by_count":85,"area":null},{"id":"2106.15013","title":"Small random initialization is akin to spectral learning: Optimization and generalization guarantees for overparameterized low-rank matrix reconstruction","created":"2021-06-28","cited_by_count":78,"area":null},{"id":"2302.01772","title":"Fixing by Mixing: A Recipe for Optimal Byzantine ML under Heterogeneity","created":"2023-02-03","cited_by_count":65,"area":null},{"id":"2109.14119","title":"Stochastic Training is Not Necessary for Generalization","created":"2021-09-29","cited_by_count":63,"area":null},{"id":"2209.09211","title":"Neural Collapse with Normalized Features: A Geometric Analysis over the Riemannian Manifold","created":"2022-09-19","cited_by_count":50,"area":null},{"id":"2102.11270","title":"Softmax Policy Gradient Methods Can Take Exponential Time to Converge","created":"2021-02-22","cited_by_count":48,"area":null},{"id":"2402.11867","title":"LoRA Training in the NTK Regime has No Spurious Local Minima","created":"2024-02-19","cited_by_count":48,"area":null},{"id":"2402.01258","title":"Transformers Learn Nonlinear Features In Context: Nonconvex Mean-field Dynamics on the Attention Landscape","created":"2024-02-02","cited_by_count":46,"area":null},{"id":"2106.14289","title":"Global Convergence of Gradient Descent for Asymmetric Low-Rank Matrix Factorization","created":"2021-06-27","cited_by_count":45,"area":null},{"id":"2507.07101","title":"Small Batch Size Training for Language Models: When Vanilla SGD Works, and Why Gradient Accumulation Is Wasteful","created":"2025-07-09","cited_by_count":41,"area":null},{"id":"2107.11228","title":"Taxonomizing local versus global structure in neural network loss landscapes","created":"2021-07-23","cited_by_count":34,"area":null},{"id":"2105.14417","title":"Overparameterization of deep ResNet: zero loss and mean-field analysis","created":"2021-05-30","cited_by_count":24,"area":null},{"id":"2202.03684","title":"Efficiently Escaping Saddle Points in Bilevel Optimization","created":"2022-02-08","cited_by_count":19,"area":null},{"id":"2310.01769","title":"How Over-Parameterization Slows Down Gradient Descent in Matrix Sensing: The Curses of Symmetry and Initialization","created":"2023-10-03","cited_by_count":18,"area":null},{"id":"2210.01212","title":"spred: Solving $L_1$ Penalty with SGD","created":"2022-10-03","cited_by_count":17,"area":null},{"id":"2210.06723","title":"Stochastic noise can be helpful for variational quantum algorithms","created":"2022-10-13","cited_by_count":16,"area":null},{"id":"2204.01368","title":"Training Fully Connected Neural Networks is $\\exists\\mathbb{R}$-Complete","created":"2022-04-04","cited_by_count":15,"area":null},{"id":"2301.06308","title":"Stability Analysis of Sharpness-Aware Minimization","created":"2023-01-16","cited_by_count":14,"area":null},{"id":"2408.09966","title":"Mask in the Mirror: Implicit Sparsification","created":"2024-08-19","cited_by_count":14,"area":null},{"id":"2110.02926","title":"On the Global Convergence of Gradient Descent for multi-layer ResNets in the mean-field regime","created":"2021-10-06","cited_by_count":12,"area":null},{"id":"2204.11516","title":"Randomly Initialized Alternating Least Squares: Fast Convergence for Matrix Sensing","created":"2022-04-25","cited_by_count":11,"area":null},{"id":"2302.01952","title":"On a continuous time model of gradient descent dynamics and instability in deep learning","created":"2023-02-03","cited_by_count":11,"area":null},{"id":"2105.10360","title":"Multi-source Learning via Completion of Block-wise Overlapping Noisy Matrices","created":"2021-05-21","cited_by_count":10,"area":null},{"id":"2205.15142","title":"Special Properties of Gradient Descent with Large Learning Rates","created":"2022-05-30","cited_by_count":10,"area":null},{"id":"2206.02041","title":"First-Order Algorithms for Min-Max Optimization in Geodesic Metric Spaces","created":"2022-06-04","cited_by_count":10,"area":null},{"id":"2212.07534","title":"Decentralized Nonconvex Optimization with Guaranteed Privacy and Accuracy","created":"2022-12-14","cited_by_count":10,"area":null},{"id":"2302.10181","title":"Exploring the Effect of Multi-step Ascent in Sharpness-Aware Minimization","created":"2023-01-27","cited_by_count":9,"area":null},{"id":"2106.02985","title":"Escaping Saddle Points Faster with Stochastic Momentum","created":"2021-06-05","cited_by_count":8,"area":null},{"id":"2107.11774","title":"SGD with a Constant Large Learning Rate Can Converge to Local Maxima","created":"2021-07-25","cited_by_count":8,"area":null},{"id":"2401.07250","title":"Stabilizing Sharpness-aware Minimization Through A Simple Renormalization Strategy","created":"2024-01-14","cited_by_count":8,"area":null},{"id":"2106.09815","title":"Escaping strict saddle points of the Moreau envelope in nonsmooth optimization","created":"2021-06-17","cited_by_count":7,"area":null},{"id":"2410.08395","title":"Nesterov acceleration in benignly non-convex landscapes","created":"2024-10-10","cited_by_count":7,"area":null},{"id":"2501.00817","title":"Hardness of Learning Fixed Parities with Neural Networks","created":"2025-01-01","cited_by_count":7,"area":null},{"id":"2210.01496","title":"Zeroth-Order Negative Curvature Finding: Escaping Saddle Points without Gradients","created":"2022-10-04","cited_by_count":6,"area":null},{"id":"2110.13265","title":"On the Second-order Convergence Properties of Random Search Methods","created":"2021-10-25","cited_by_count":5,"area":null},{"id":"2208.02083","title":"Gradient descent provably escapes saddle points in the training of shallow ReLU networks","created":"2022-08-03","cited_by_count":5,"area":null},{"id":"2208.10025","title":"Simple and Optimal Stochastic Gradient Methods for Nonsmooth Nonconvex Optimization","created":"2022-08-22","cited_by_count":5,"area":null},{"id":"2209.09162","title":"On the Theoretical Properties of Noise Correlation in Stochastic Optimization","created":"2022-09-19","cited_by_count":5,"area":null},{"id":"2302.04753","title":"Efficient displacement convex optimization with particle gradient descent","created":"2023-02-09","cited_by_count":5,"area":null},{"id":"2102.02837","title":"Escaping Saddle Points for Nonsmooth Weakly Convex Functions via Perturbed Proximal Algorithms","created":"2021-02-04","cited_by_count":4,"area":null},{"id":"2205.00832","title":"Gradient Descent, Stochastic Optimization, and Other Tales","created":"2022-05-02","cited_by_count":4,"area":null},{"id":"2209.15130","title":"Nonconvex Matrix Factorization is Geodesically Convex: Global Landscape Analysis for Fixed-rank Matrix Optimization From a Riemannian Perspective","created":"2022-09-29","cited_by_count":4,"area":null},{"id":"2212.09396","title":"Rank-1 Matrix Completion with Gradient Descent and Small Random Initialization","created":"2022-12-19","cited_by_count":4,"area":null},{"id":"2306.12534","title":"Memory-Query Tradeoffs for Randomized Convex Optimization","created":"2023-06-21","cited_by_count":4,"area":null},{"id":"2310.19059","title":"Escaping Saddle Points in Heterogeneous Federated Learning via Distributed SGD with Communication Compression","created":"2023-10-29","cited_by_count":4,"area":null},{"id":"2603.02064","title":"Never Saddle for Reparameterized Steepest Descent as Mirror Flow","created":"2026-03-02","cited_by_count":4,"area":null},{"id":"2103.09424","title":"Escaping Saddle Points in Distributed Newton's Method with Communication Efficiency and Byzantine Resilience","created":"2021-03-17","cited_by_count":3,"area":null},{"id":"2106.03763","title":"Vanishing Curvature and the Power of Adaptive Methods in Randomly Initialized Deep Networks","created":"2021-06-07","cited_by_count":3,"area":null},{"id":"2408.13276","title":"Non-convex matrix sensing: Breaking the quadratic rank barrier in the sample complexity","created":"2024-08-20","cited_by_count":3,"area":null},{"id":"2209.03705","title":"Losing momentum in continuous-time stochastic optimisation","created":"2022-09-08","cited_by_count":2,"area":null},{"id":"2212.12086","title":"Eigenvalue initialisation and regularisation for Koopman autoencoders","created":"2022-12-23","cited_by_count":2,"area":null},{"id":"2401.03214","title":"Understanding Representation Learnability of Nonlinear Self-Supervised Learning","created":"2024-01-06","cited_by_count":2,"area":null},{"id":"2402.09807","title":"Two trust region type algorithms for solving nonconvex-strongly concave minimax problems","created":"2024-02-15","cited_by_count":2,"area":null},{"id":"2402.18311","title":"Escaping Local Optima in Global Placement","created":"2024-02-28","cited_by_count":2,"area":null},{"id":"2403.10423","title":"Quantization Avoids Saddle Points in Distributed Optimization","created":"2024-03-15","cited_by_count":2,"area":null},{"id":"2509.16974","title":"Hessian-guided Perturbed Wasserstein Gradient Flows for Escaping Saddle Points","created":"2025-09-21","cited_by_count":2,"area":null},{"id":"2103.00075","title":"Noisy Truncated SGD: Optimization and Generalization","created":"2021-02-26","cited_by_count":1,"area":null},{"id":"2105.10090","title":"Escaping Saddle Points with Compressed SGD","created":"2021-05-21","cited_by_count":1,"area":null},{"id":"2110.13144","title":"Faster Perturbed Stochastic Gradient Methods for Finding Local Minima","created":"2021-10-25","cited_by_count":1,"area":null},{"id":"2302.07862","title":"Almost Sure Saddle Avoidance of Stochastic Gradient Methods without the Bounded Gradient Assumption","created":"2023-02-15","cited_by_count":1,"area":null},{"id":"2305.18188","title":"Understanding Predictive Coding as an Adaptive Trust-Region Method","created":"2023-05-29","cited_by_count":1,"area":null},{"id":"2410.24050","title":"A Mechanistic Study of Transformers Training Dynamics","created":"2024-10-31","cited_by_count":1,"area":null},{"id":"2101.02625","title":"Boundary Conditions for Linear Exit Time Gradient Trajectories Around Saddle Points: Analysis and Algorithm","created":"2021-01-07","cited_by_count":0,"area":null},{"id":"2205.13753","title":"HOUDINI: Escaping from Moderately Constrained Saddles","created":"2022-05-27","cited_by_count":0,"area":null},{"id":"2211.00866","title":"Gradient Descent and the Power Method: Exploiting their connection to find the leftmost eigen-pair and escape saddle points","created":"2022-11-02","cited_by_count":0,"area":null},{"id":"2212.03765","title":"Generalized Gradient Flows with Provable Fixed-Time Convergence and Fast Evasion of Non-Degenerate Saddle Points","created":"2022-12-07","cited_by_count":0,"area":null},{"id":"2307.16099","title":"On Neural Network approximation of ideal adversarial attack and convergence of adversarial training","created":"2023-07-30","cited_by_count":0,"area":null},{"id":"2403.09646","title":"On Unsupervised Image-to-image translation and GAN stability","created":"2023-10-18","cited_by_count":0,"area":null},{"id":"2409.11713","title":"From exponential to finite/fixed-time stability: Applications to optimization","created":"2024-09-18","cited_by_count":0,"area":null},{"id":"2411.15769","title":"Gradient Norm Regularization Second-Order Algorithms for Solving Nonconvex-Strongly Concave Minimax Problems","created":"2024-11-24","cited_by_count":0,"area":null},{"id":"2605.01288","title":"A Theory of Saddle Escape in Deep Nonlinear Networks","created":"2026-05-02","cited_by_count":0,"area":null},{"id":"2609.13057","title":"Benign Loss Landscapes Can Coexist with Worst-Case Hardness","created":"2026-09-11","cited_by_count":0,"area":null},{"id":"2610.01697","title":"Learning PDE Dynamics between Submanifolds Using Green's Observation Operators","created":"2026-10-01","cited_by_count":0,"area":null}],"next_cursor":null}