{"total":475,"rows":[{"id":"2305.17333","title":"Fine-Tuning Language Models with Just Forward Passes","created":"2023-05-27","cited_by_count":345,"area":null},{"id":"2109.00650","title":"Dash: Semi-Supervised Learning with Dynamic Thresholding","created":"2021-09-01","cited_by_count":148,"area":null},{"id":"2206.06232","title":"Towards Understanding Sharpness-Aware Minimization","created":"2022-06-13","cited_by_count":146,"area":null},{"id":"2310.17022","title":"Controlled Decoding from Language Models","created":"2023-10-25","cited_by_count":135,"area":null},{"id":"2105.14368","title":"Fit without fear: remarkable mathematical phenomena of deep learning through the prism of interpolation","created":"2021-05-29","cited_by_count":132,"area":null},{"id":"2102.07053","title":"Linear Convergence in Federated Learning: Tackling Client Heterogeneity and Sparse Gradients","created":"2021-02-14","cited_by_count":123,"area":null},{"id":"2201.07443","title":"On the Convergence Rates of Policy Gradient Methods","created":"2022-01-19","cited_by_count":119,"area":null},{"id":"2104.11375","title":"Decentralized Federated Averaging","created":"2021-04-23","cited_by_count":114,"area":null},{"id":"2402.01868","title":"Challenges in Training PINNs: A Loss Landscape Perspective","created":"2024-02-02","cited_by_count":113,"area":null},{"id":"2209.08709","title":"BOME! Bilevel Optimization Made Easy: A Simple First-Order Approach","created":"2022-09-19","cited_by_count":105,"area":null},{"id":"2102.12660","title":"Distributionally Robust Federated Averaging","created":"2021-02-25","cited_by_count":104,"area":null},{"id":"2302.05185","title":"On Penalty-based Bilevel Gradient Descent Method","created":"2023-02-10","cited_by_count":87,"area":null},{"id":"2206.07085","title":"Understanding the Generalization Benefit of Normalization Layers: Sharpness Reduction","created":"2022-06-14","cited_by_count":86,"area":null},{"id":"2302.12022","title":"DoG is SGD's Best Friend: A Parameter-Free Dynamic Step Size Schedule","created":"2023-02-08","cited_by_count":85,"area":null},{"id":"2202.06386","title":"Improved analysis for a proximal algorithm for sampling","created":"2022-02-13","cited_by_count":73,"area":null},{"id":"2505.13416","title":"Gluon: Making Muon & Scion Great Again! (Bridging Theory and Practice of LMO-based Optimizers for LLMs)","created":"2025-05-19","cited_by_count":72,"area":null},{"id":"2205.07344","title":"Policy Gradient Method For Robust Reinforcement Learning","created":"2022-05-15","cited_by_count":69,"area":null},{"id":"2202.05214","title":"Towards a Theory of Non-Log-Concave Sampling: First-Order Stationarity Guarantees for Langevin Monte Carlo","created":"2022-02-10","cited_by_count":63,"area":null},{"id":"2106.12379","title":"AC/DC: Alternating Compressed/DeCompressed Training of Deep Neural Networks","created":"2021-06-23","cited_by_count":59,"area":null},{"id":"2101.06905","title":"Blockchain Assisted Decentralized Federated Learning (BLADE-FL): Performance Analysis and Resource Allocation","created":"2021-01-18","cited_by_count":56,"area":null},{"id":"2410.13835","title":"Active-Dormant Attention Heads: Mechanistically Demystifying Extreme-Token Phenomena in LLMs","created":"2024-10-17","cited_by_count":56,"area":null},{"id":"2101.08286","title":"Can stable and accurate neural networks be computed? -- On the barriers of deep learning and Smale's 18th problem","created":"2021-01-20","cited_by_count":53,"area":null},{"id":"2106.12529","title":"Who Leads and Who Follows in Strategic Classification?","created":"2021-06-23","cited_by_count":53,"area":null},{"id":"2209.05423","title":"Statistical Learning Theory for Control: A Finite Sample Perspective","created":"2022-09-12","cited_by_count":53,"area":null},{"id":"2403.12350","title":"Friendly Sharpness-Aware Minimization","created":"2024-03-19","cited_by_count":53,"area":null},{"id":"2309.01753","title":"On Penalty Methods for Nonconvex Bilevel Optimization and First-Order Stochastic Approximation","created":"2023-09-04","cited_by_count":51,"area":null},{"id":"2112.05604","title":"Faster Single-loop Algorithms for Minimax Optimization without Strong Concavity","created":"2021-12-10","cited_by_count":49,"area":null},{"id":"2102.02410","title":"A Local Convergence Theory for Mildly Over-Parameterized Two-Layer Neural Network","created":"2021-02-04","cited_by_count":47,"area":null},{"id":"2105.06072","title":"Leveraging Non-uniformity in First-order Non-convex Optimization","created":"2021-05-13","cited_by_count":44,"area":null},{"id":"2107.00052","title":"Stochastic Gradient Descent-Ascent and Consensus Optimization for Smooth Games: Convergence Analysis under Expected Co-coercivity","created":"2021-06-30","cited_by_count":43,"area":null},{"id":"2109.06141","title":"On Tilted Losses in Machine Learning: Theory and Applications","created":"2021-09-13","cited_by_count":43,"area":null},{"id":"2103.12024","title":"Stability and Deviation Optimal Risk Bounds with Convergence Rate $O(1/n)$","created":"2021-03-22","cited_by_count":42,"area":null},{"id":"2301.11500","title":"Understanding Incremental Learning of Gradient Descent: A Fine-grained Analysis of Matrix Sensing","created":"2023-01-27","cited_by_count":42,"area":null},{"id":"2102.13152","title":"Local Stochastic Gradient Descent Ascent: Convergence Analysis and Communication Efficiency","created":"2021-02-25","cited_by_count":40,"area":null},{"id":"2305.16296","title":"A Guide Through the Zoo of Biased SGD","created":"2023-05-25","cited_by_count":40,"area":null},{"id":"2105.01650","title":"Stochastic gradient descent with noise of machine learning type. Part I: Discrete time analysis","created":"2021-05-04","cited_by_count":39,"area":null},{"id":"2301.00712","title":"On Finding Small Hyper-Gradients in Bilevel Optimization: Hardness Results and Improved Analysis","created":"2023-01-02","cited_by_count":39,"area":null},{"id":"2408.10147","title":"In-Context Learning with Representations: Contextual Generalization of Trained Transformers","created":"2024-08-19","cited_by_count":38,"area":null},{"id":"2203.04850","title":"Federated Minimax Optimization: Improved Convergence Analyses and Algorithms","created":"2022-03-09","cited_by_count":37,"area":null},{"id":"2105.03793","title":"Stability and Generalization of Stochastic Gradient Methods for Minimax Problems","created":"2021-05-08","cited_by_count":36,"area":null},{"id":"2111.12292","title":"Improved Fine-Tuning by Better Leveraging Pre-Training Data","created":"2021-11-24","cited_by_count":36,"area":null},{"id":"2102.07346","title":"On the Theory of Implicit Deep Learning: Global Convergence with Implicit Layers","created":"2021-02-15","cited_by_count":35,"area":null},{"id":"2102.07432","title":"Fast and accurate optimization on the orthogonal manifold without retraction","created":"2021-02-15","cited_by_count":35,"area":null},{"id":"2102.06704","title":"Proximal and Federated Random Reshuffling","created":"2021-02-12","cited_by_count":33,"area":null},{"id":"2106.08502","title":"Averaging on the Bures-Wasserstein manifold: dimension-free convergence of gradient descent","created":"2021-06-16","cited_by_count":32,"area":null},{"id":"2302.03109","title":"On the Convergence of Federated Averaging with Cyclic Client Participation","created":"2023-02-06","cited_by_count":31,"area":null},{"id":"2509.15816","title":"On the Convergence of Muon and Beyond","created":"2025-09-19","cited_by_count":31,"area":null},{"id":"2102.09385","title":"Convergence of stochastic gradient descent schemes for Lojasiewicz-landscapes","created":"2021-02-16","cited_by_count":28,"area":null},{"id":"2203.06735","title":"Private Non-Convex Federated Learning Without a Trusted Server","created":"2022-03-13","cited_by_count":26,"area":null},{"id":"2206.04804","title":"Theoretical Error Performance Analysis for Variational Quantum Circuit Based Functional Regression","created":"2022-06-08","cited_by_count":26,"area":null},{"id":"2305.15287","title":"The Crucial Role of Normalization in Sharpness-Aware Minimization","created":"2023-05-24","cited_by_count":26,"area":null},{"id":"2309.13591","title":"Robust Distributed Learning: Tight Error Bounds and Breakdown Point under Data Heterogeneity","created":"2023-09-24","cited_by_count":26,"area":null},{"id":"2310.09639","title":"DPZero: Private Fine-Tuning of Language Models without Backpropagation","created":"2023-10-14","cited_by_count":26,"area":null},{"id":"2105.00872","title":"Convergence Analysis and System Design for Federated Learning over Wireless Networks","created":"2021-04-30","cited_by_count":25,"area":null},{"id":"2109.14756","title":"A Two-Time-Scale Stochastic Optimization Framework with Applications in Control and Reinforcement Learning","created":"2021-09-29","cited_by_count":24,"area":null},{"id":"2203.16462","title":"Convergence of gradient descent for deep neural networks","created":"2022-03-30","cited_by_count":24,"area":null},{"id":"2302.04787","title":"On the Privacy-Robustness-Utility Trilemma in Distributed Learning","created":"2023-02-09","cited_by_count":24,"area":null},{"id":"2410.10373","title":"Sharpness-Aware Minimization Efficiently Selects Flatter Minima Late in Training","created":"2024-10-14","cited_by_count":24,"area":null},{"id":"2107.10110","title":"On the Convergence of Prior-Guided Zeroth-Order Optimization Algorithms","created":"2021-07-21","cited_by_count":23,"area":null},{"id":"2201.09046","title":"Differentially Private SGDA for Minimax Problems","created":"2022-01-22","cited_by_count":23,"area":null},{"id":"2210.07863","title":"Revisiting Optimal Convergence Rate for Smooth and Non-convex Stochastic Decentralized Optimization","created":"2022-10-14","cited_by_count":23,"area":null},{"id":"2305.13404","title":"Improving Convergence and Generalization Using Parameter Symmetries","created":"2023-05-22","cited_by_count":23,"area":null},{"id":"2108.02040","title":"Convergence of gradient descent for learning linear neural networks","created":"2021-08-04","cited_by_count":22,"area":null},{"id":"2305.15349","title":"On the Convergence of Black-Box Variational Inference","created":"2023-05-24","cited_by_count":22,"area":null},{"id":"2306.02601","title":"Aiming towards the minimizers: fast convergence of SGD for overparametrized problems","created":"2023-06-05","cited_by_count":22,"area":null},{"id":"2106.13792","title":"Proxy Convexity: A Unified Framework for the Analysis of Neural Networks Trained by Gradient Descent","created":"2021-06-25","cited_by_count":21,"area":null},{"id":"2402.00949","title":"Geometry of Polynomial Neural Networks","created":"2024-02-01","cited_by_count":21,"area":null},{"id":"2506.00362","title":"FSNet: Feasibility-Seeking Neural Network for Constrained Optimization with Guarantees","created":"2025-05-31","cited_by_count":21,"area":null},{"id":"2106.08882","title":"Robust Training in High Dimensions via Block Coordinate Geometric Median Descent","created":"2021-06-16","cited_by_count":20,"area":null},{"id":"2111.12050","title":"Simple Stochastic and Online Gradient Descent Algorithms for Pairwise Learning","created":"2021-11-23","cited_by_count":20,"area":null},{"id":"2205.12856","title":"Stochastic Second-Order Methods Improve Best-Known Sample Complexity of SGD for Gradient-Dominated Function","created":"2022-05-25","cited_by_count":20,"area":null},{"id":"2206.00833","title":"Finite-Time Analysis of Entropy-Regularized Neural Natural Actor-Critic Algorithm","created":"2022-06-02","cited_by_count":20,"area":null},{"id":"2207.00391","title":"A Theoretical Analysis of the Learning Dynamics under Class Imbalance","created":"2022-07-01","cited_by_count":20,"area":null},{"id":"2405.06979","title":"Robust Semi-supervised Learning by Wisely Leveraging Open-set Data","created":"2024-05-11","cited_by_count":20,"area":null},{"id":"2106.04759","title":"Communication-efficient SGD: From Local SGD to One-Shot Averaging","created":"2021-06-09","cited_by_count":19,"area":null},{"id":"2204.12446","title":"Beyond Lipschitz: Sharp Generalization and Excess Risk Bounds for Full-Batch GD","created":"2022-04-26","cited_by_count":19,"area":null},{"id":"2402.07114","title":"Towards Quantifying the Preconditioning Effect of Adam","created":"2024-02-11","cited_by_count":19,"area":null},{"id":"2102.04653","title":"Proximal Gradient Descent-Ascent: Variable Convergence under K{\\L} Geometry","created":"2021-02-09","cited_by_count":18,"area":null},{"id":"2205.13746","title":"Regularized Gradient Descent Ascent for Two-Player Zero-Sum Markov Games","created":"2022-05-27","cited_by_count":18,"area":null},{"id":"2405.01031","title":"The Privacy Power of Correlated Noise in Decentralized Learning","created":"2024-05-02","cited_by_count":18,"area":null},{"id":"2110.05645","title":"A global convergence theory for deep ReLU implicit networks via over-parameterization","created":"2021-10-11","cited_by_count":17,"area":null},{"id":"2202.00308","title":"PAGE-PG: A Simple and Loopless Variance-Reduced Policy Gradient Method with Probabilistic Gradient Estimation","created":"2022-02-01","cited_by_count":17,"area":null},{"id":"2202.07530","title":"Optimal Algorithms for Stochastic Multi-Level Compositional Optimization","created":"2022-02-15","cited_by_count":17,"area":null},{"id":"2205.04180","title":"EF-BV: A Unified Theory of Error Feedback and Variance Reduction Mechanisms for Biased and Unbiased Compression in Distributed Optimization","created":"2022-05-09","cited_by_count":17,"area":null},{"id":"2403.04081","title":"Directional Smoothness and Gradient Methods: Convergence and Adaptivity","created":"2024-03-06","cited_by_count":17,"area":null},{"id":"2101.04968","title":"Learning with Gradient Descent and Weakly Convex Losses","created":"2021-01-13","cited_by_count":16,"area":null},{"id":"2110.11442","title":"Towards Noise-adaptive, Problem-adaptive (Accelerated) Stochastic Gradient Descent","created":"2021-10-21","cited_by_count":16,"area":null},{"id":"2207.08540","title":"Multi-block-Single-probe Variance Reduced Estimator for Coupled Compositional Optimization","created":"2022-07-18","cited_by_count":16,"area":null},{"id":"2302.08766","title":"A Lower Bound and a Near-Optimal Algorithm for Bilevel Empirical Risk Minimization","created":"2023-02-17","cited_by_count":16,"area":null},{"id":"2307.08352","title":"Zero-th Order Algorithm for Softmax Attention Optimization","created":"2023-07-17","cited_by_count":16,"area":null},{"id":"2310.11914","title":"A connection between Tempering and Entropic Mirror Descent","created":"2023-10-18","cited_by_count":16,"area":null},{"id":"2406.18038","title":"MT2ST: Adaptive Multi-Task to Single-Task Learning","created":"2024-06-26","cited_by_count":16,"area":null},{"id":"2411.10830","title":"One-Layer Transformer Provably Learns One-Nearest Neighbor In Context","created":"2024-11-16","cited_by_count":16,"area":null},{"id":"2312.02828","title":"Convergence Rates for Stochastic Approximation: Biased Noise with Unbounded Variance, and Applications","created":"2023-12-05","cited_by_count":15,"area":null},{"id":"2410.11778","title":"On the Training Convergence of Transformers for In-Context Classification of Gaussian Mixtures","created":"2024-10-15","cited_by_count":15,"area":null},{"id":"2206.05891","title":"Anchor Sampling for Federated Learning with Partial Client Participation","created":"2022-06-13","cited_by_count":14,"area":null},{"id":"2211.16162","title":"Scalable Hierarchical Over-the-Air Federated Learning","created":"2022-11-29","cited_by_count":14,"area":null},{"id":"2302.11051","title":"Fusion of Global and Local Knowledge for Personalized Federated Learning","created":"2023-02-21","cited_by_count":14,"area":null},{"id":"2303.03944","title":"On Momentum-Based Gradient Methods for Bilevel Optimization with Nonconvex Lower-Level","created":"2023-03-07","cited_by_count":14,"area":null},{"id":"2303.04274","title":"Amplitude-Varying Perturbation for Balancing Privacy and Utility in Federated Learning","created":"2023-03-07","cited_by_count":14,"area":null}],"next_cursor":"100"}