{"total":1088,"rows":[{"id":"2103.00065","title":"Gradient Descent on Neural Networks Typically Occurs at the Edge of Stability","created":"2021-02-26","cited_by_count":427,"area":null},{"id":"2105.02375","title":"A Geometric Analysis of Neural Collapse with Unconstrained Features","created":"2021-05-06","cited_by_count":227,"area":null},{"id":"2101.11203","title":"Achieving Linear Speedup with Partial Worker Participation in Non-IID Federated Learning","created":"2021-01-27","cited_by_count":193,"area":null},{"id":"2106.06639","title":"Federated Learning with Buffered Asynchronous Aggregation","created":"2021-06-11","cited_by_count":190,"area":null},{"id":"2101.12699","title":"Exploring Deep Neural Networks via Layer-Peeled Model: Minority Collapse in Imbalanced Training","created":"2021-01-29","cited_by_count":166,"area":null},{"id":"2111.01549","title":"Overcoming Catastrophic Forgetting in Incremental Few-Shot Learning by Finding Flat Minima","created":"2021-10-30","cited_by_count":118,"area":null},{"id":"2204.03809","title":"Federated Learning with Partial Model Personalization","created":"2022-04-08","cited_by_count":114,"area":null},{"id":"2103.05180","title":"An Introduction to Deep Generative Modeling","created":"2021-03-09","cited_by_count":104,"area":null},{"id":"2107.04562","title":"The Bayesian Learning Rule","created":"2021-07-09","cited_by_count":99,"area":null},{"id":"2201.13409","title":"A framework for bilevel optimization that enables stochastic and global variance reduction algorithms","created":"2022-01-31","cited_by_count":99,"area":null},{"id":"2103.04715","title":"Bayesian imaging using Plug & Play priors: when Langevin meets Tweedie","created":"2021-03-08","cited_by_count":97,"area":null},{"id":"2203.01238","title":"On the Optimization Landscape of Neural Collapse under MSE Loss: Global Optimality with Unconstrained Features","created":"2022-03-02","cited_by_count":97,"area":null},{"id":"2105.05883","title":"Clustered Sampling: Low-Variance and Improved Representativity for Clients Selection in Federated Learning","created":"2021-05-12","cited_by_count":93,"area":null},{"id":"2403.14123","title":"AI and Memory Wall","created":"2024-03-21","cited_by_count":92,"area":null},{"id":"2206.07136","title":"Automatic Clipping: Differentially Private Deep Learning Made Easier and Stronger","created":"2022-06-14","cited_by_count":88,"area":null},{"id":"2206.04920","title":"Fisher SAM: Information Geometry and Sharpness Aware Minimisation","created":"2022-06-10","cited_by_count":84,"area":null},{"id":"2305.01588","title":"Revisiting Gradient Clipping: Stochastic bias and tight convergence guarantees","created":"2023-05-02","cited_by_count":83,"area":null},{"id":"2206.08307","title":"Sharper Convergence Guarantees for Asynchronous SGD for Distributed and Federated Learning","created":"2022-06-16","cited_by_count":79,"area":null},{"id":"2202.00661","title":"When Do Flat Minima Optimizers Work?","created":"2022-02-01","cited_by_count":78,"area":null},{"id":"2309.06497","title":"A Distributed Data-Parallel PyTorch Implementation of the Distributed Shampoo Optimizer for Training Neural Networks At-Scale","created":"2023-09-12","cited_by_count":77,"area":null},{"id":"2107.11433","title":"A general sample complexity analysis of vanilla policy gradient","created":"2021-07-23","cited_by_count":76,"area":null},{"id":"2210.05177","title":"Make Sharpness-Aware Minimization Stronger: A Sparsified Perturbation Approach","created":"2022-10-11","cited_by_count":70,"area":null},{"id":"2102.04761","title":"Quasi-Global Momentum: Accelerating Decentralized Deep Learning on Heterogeneous Data","created":"2021-02-09","cited_by_count":69,"area":null},{"id":"2107.06925","title":"Chimera: Efficiently Training Large-Scale Neural Networks with Bidirectional Pipelines","created":"2021-07-14","cited_by_count":68,"area":null},{"id":"2305.19229","title":"FedDisco: Federated Learning with Discrepancy-Aware Collaboration","created":"2023-05-30","cited_by_count":67,"area":null},{"id":"2112.14038","title":"DAS-PINNs: A deep adaptive sampling method for solving high-dimensional partial differential equations","created":"2021-12-28","cited_by_count":66,"area":null},{"id":"2302.01772","title":"Fixing by Mixing: A Recipe for Optimal Byzantine ML under Heterogeneity","created":"2023-02-03","cited_by_count":65,"area":null},{"id":"2102.04487","title":"Adaptive Quantization of Model Updates for Communication-Efficient Federated Learning","created":"2021-02-08","cited_by_count":63,"area":null},{"id":"2105.13462","title":"On Linear Stability of SGD and Input-Smoothness of Neural Networks","created":"2021-05-27","cited_by_count":63,"area":null},{"id":"2205.12173","title":"Byzantine Machine Learning Made Easy by Resilient Averaging of Momentums","created":"2022-05-24","cited_by_count":63,"area":null},{"id":"2207.12678","title":"Analyzing Sharpness along GD Trajectory: Progressive Sharpening and Edge of Stability","created":"2022-07-26","cited_by_count":63,"area":null},{"id":"2104.14129","title":"ActNN: Reducing Training Memory Footprint via 2-Bit Activation Compressed Training","created":"2021-04-29","cited_by_count":61,"area":null},{"id":"2104.14937","title":"Federated Learning with Fair Averaging","created":"2021-04-30","cited_by_count":61,"area":null},{"id":"2202.05791","title":"The Power of Adaptivity in SGD: Self-Tuning Step Sizes with Unbounded Gradients and Affine Variance","created":"2022-02-11","cited_by_count":61,"area":null},{"id":"2201.09531","title":"Communication-Efficient Stochastic Zeroth-Order Optimization for Federated Learning","created":"2022-01-24","cited_by_count":59,"area":null},{"id":"2102.01567","title":"A Lyapunov Theory for Finite-Sample Guarantees of Asynchronous Q-Learning and TD-Learning Variants","created":"2021-02-02","cited_by_count":57,"area":null},{"id":"2206.10654","title":"On the Maximum Hessian Eigenvalue and Generalization","created":"2022-06-21","cited_by_count":57,"area":null},{"id":"2107.12048","title":"Decentralized Federated Learning: Balancing Communication and Computing Costs","created":"2021-07-26","cited_by_count":55,"area":null},{"id":"2306.00204","title":"Toward Understanding Why Adam Converges Faster Than SGD for Transformers","created":"2023-05-31","cited_by_count":53,"area":null},{"id":"2310.16608","title":"Performative Prediction: Past and Future","created":"2023-10-25","cited_by_count":53,"area":null},{"id":"2110.03677","title":"Large Learning Rate Tames Homogeneity: Convergence and Balancing Effect","created":"2021-10-07","cited_by_count":50,"area":null},{"id":"2102.04828","title":"Consensus Control for Decentralized Deep Learning","created":"2021-02-09","cited_by_count":49,"area":null},{"id":"2410.13849","title":"From Gradient Clipping to Normalization for Heavy Tailed SGD","created":"2024-10-17","cited_by_count":49,"area":null},{"id":"2103.14272","title":"Hierarchical Federated Learning with Quantization: Convergence Analysis and System Design","created":"2021-03-26","cited_by_count":47,"area":null},{"id":"2302.04083","title":"Improving the Model Consistency of Decentralized Federated Learning","created":"2023-02-08","cited_by_count":45,"area":null},{"id":"2106.10435","title":"STEM: A Stochastic Two-Sided Momentum Algorithm Achieving Near-Optimal Sample and Communication Complexities for Federated Learning","created":"2021-06-19","cited_by_count":43,"area":null},{"id":"2107.00052","title":"Stochastic Gradient Descent-Ascent and Consensus Optimization for Smooth Games: Convergence Analysis under Expected Co-coercivity","created":"2021-06-30","cited_by_count":43,"area":null},{"id":"2302.01735","title":"Rethinking Semi-Supervised Medical Image Segmentation: A Variance-Reduction Perspective","created":"2023-02-03","cited_by_count":43,"area":null},{"id":"2304.14647","title":"An Adaptive Policy to Employ Sharpness-Aware Minimization","created":"2023-04-28","cited_by_count":43,"area":null},{"id":"2108.09875","title":"Anarchic Federated Learning","created":"2021-08-23","cited_by_count":42,"area":null},{"id":"2412.19529","title":"Nonconvex Stochastic Optimization under Heavy-Tailed Noises: Optimal Convergence without Gradient Clipping","created":"2024-12-27","cited_by_count":42,"area":null},{"id":"2207.04475","title":"Finite-time High-probability Bounds for Polyak-Ruppert Averaged Iterates of Linear Stochastic Approximation","created":"2022-07-10","cited_by_count":41,"area":null},{"id":"2302.06570","title":"Beyond Uniform Smoothness: A Stopped Analysis of Adaptive SGD","created":"2023-02-13","cited_by_count":41,"area":null},{"id":"2304.09121","title":"Fast Neural Scene Flow","created":"2023-04-18","cited_by_count":41,"area":null},{"id":"2502.04664","title":"Implicit Bias of Spectral Descent and Muon on Multiclass Separable Data","created":"2025-02-07","cited_by_count":41,"area":null},{"id":"2305.16296","title":"A Guide Through the Zoo of Biased SGD","created":"2023-05-25","cited_by_count":40,"area":null},{"id":"2105.01650","title":"Stochastic gradient descent with noise of machine learning type. Part I: Discrete time analysis","created":"2021-05-04","cited_by_count":39,"area":null},{"id":"2110.08450","title":"Accelerating Training and Inference of Graph Neural Networks with Fast Sampling and Pipelining","created":"2021-10-16","cited_by_count":39,"area":null},{"id":"2302.01107","title":"A Survey on Efficient Training of Transformers","created":"2023-02-02","cited_by_count":39,"area":null},{"id":"2305.00241","title":"When Deep Learning Meets Polyhedral Theory: A Survey","created":"2023-04-29","cited_by_count":39,"area":null},{"id":"2302.14428","title":"Stochastic Gradient Descent under Markovian Sampling Schemes","created":"2023-02-28","cited_by_count":38,"area":null},{"id":"2203.04850","title":"Federated Minimax Optimization: Improved Convergence Analyses and Algorithms","created":"2022-03-09","cited_by_count":37,"area":null},{"id":"2209.07529","title":"On the Soft-Subnetwork for Few-shot Class Incremental Learning","created":"2022-09-15","cited_by_count":37,"area":null},{"id":"2301.12832","title":"Deep networks for system identification: a Survey","created":"2023-01-30","cited_by_count":37,"area":null},{"id":"2309.15639","title":"Enhancing Sharpness-Aware Optimization Through Variance Suppression","created":"2023-09-27","cited_by_count":37,"area":null},{"id":"2206.08330","title":"Compressed-VFL: Communication-Efficient Learning with Vertically Partitioned Data","created":"2022-06-16","cited_by_count":36,"area":null},{"id":"2304.05398","title":"Forward-backward Gaussian variational inference via JKO in the Bures-Wasserstein Space","created":"2023-04-10","cited_by_count":36,"area":null},{"id":"2601.19156","title":"Convergence of Muon with Newton-Schulz","created":"2026-01-27","cited_by_count":36,"area":null},{"id":"2102.04396","title":"SGD in the Large: Average-case Analysis, Asymptotics, and Stepsize Criticality","created":"2021-02-08","cited_by_count":35,"area":null},{"id":"2108.09103","title":"Mobility-Aware Cluster Federated Learning in Hierarchical Wireless Networks","created":"2021-08-20","cited_by_count":35,"area":null},{"id":"2202.04295","title":"On Almost Sure Convergence Rates of Stochastic Gradient Methods","created":"2022-02-09","cited_by_count":35,"area":null},{"id":"2305.16284","title":"DoWG Unleashed: An Efficient Universal Parameter-Free Gradient Descent Method","created":"2023-05-25","cited_by_count":35,"area":null},{"id":"2502.15938","title":"Straight to Zero: Why Linearly Decaying the Learning Rate to Zero Works Best for LLMs","created":"2025-02-21","cited_by_count":35,"area":null},{"id":"2202.07178","title":"Federated Learning with Sparsified Model Perturbation: Improving Accuracy under Client-Level Differential Privacy","created":"2022-02-15","cited_by_count":33,"area":null},{"id":"2303.11242","title":"Make Landscape Flatter in Differentially Private Federated Learning","created":"2023-03-20","cited_by_count":33,"area":null},{"id":"2204.04452","title":"Refined Convergence and Topology Learning for Decentralized SGD with Heterogeneous Data","created":"2022-04-09","cited_by_count":32,"area":null},{"id":"2102.01583","title":"The Min-Max Complexity of Distributed Stochastic Convex Optimization with Intermittent Communication","created":"2021-02-02","cited_by_count":31,"area":null},{"id":"2106.02305","title":"Local Adaptivity in Federated Learning: Convergence and Consistency","created":"2021-06-04","cited_by_count":31,"area":null},{"id":"2110.03128","title":"On the Generalization of Models Trained with SGD: Information-Theoretic Bounds and Implications","created":"2021-10-07","cited_by_count":31,"area":null},{"id":"2206.11357","title":"GACT: Activation Compressed Training for Generic Network Architectures","created":"2022-06-22","cited_by_count":31,"area":null},{"id":"2106.02743","title":"SpreadGNN: Serverless Multi-task Federated Learning for Graph Neural Networks","created":"2021-06-04","cited_by_count":30,"area":null},{"id":"2109.05872","title":"Byzantine-robust Federated Learning through Collaborative Malicious Gradient Filtering","created":"2021-09-13","cited_by_count":30,"area":null},{"id":"2111.00173","title":"Dynamic Differential-Privacy Preserving SGD","created":"2021-10-30","cited_by_count":30,"area":null},{"id":"2206.01299","title":"Fine-tuning Language Models over Slow Networks using Activation Compression with Guarantees","created":"2022-06-02","cited_by_count":30,"area":null},{"id":"2302.08783","title":"SGD with AdaGrad Stepsizes: Full Adaptivity with High Probability to Unknown Parameters, Unbounded Gradients and Affine Variance","created":"2023-02-17","cited_by_count":30,"area":null},{"id":"2311.03252","title":"Parameter-Agnostic Optimization under Relaxed Smoothness","created":"2023-11-06","cited_by_count":29,"area":null},{"id":"2103.11181","title":"Adaptive deep density approximation for Fokker-Planck equations","created":"2021-03-20","cited_by_count":28,"area":null},{"id":"2104.09623","title":"The mixed deep energy method for resolving concentration features in finite strain hyperelasticity","created":"2021-04-15","cited_by_count":28,"area":null},{"id":"2102.00236","title":"Parameter-free Stochastic Optimization of Variationally Coherent Functions","created":"2021-01-30","cited_by_count":27,"area":null},{"id":"2106.01257","title":"Tight High Probability Bounds for Linear Stochastic Approximation with Fixed Stepsize","created":"2021-06-02","cited_by_count":27,"area":null},{"id":"2205.13891","title":"Transformers from an Optimization Perspective","created":"2022-05-27","cited_by_count":27,"area":null},{"id":"2102.03401","title":"Federated Learning on the Road: Autonomous Controller Design for Connected and Autonomous Vehicles","created":"2021-02-05","cited_by_count":26,"area":null},{"id":"2102.09318","title":"Finite-Sample Analysis of Off-Policy Natural Actor-Critic Algorithm","created":"2021-02-18","cited_by_count":26,"area":null},{"id":"2112.10769","title":"Accurate Neural Training with 4-bit Matrix Multiplications at Standard Formats","created":"2021-12-19","cited_by_count":26,"area":null},{"id":"2309.13591","title":"Robust Distributed Learning: Tight Error Bounds and Breakdown Point under Data Heterogeneity","created":"2023-09-24","cited_by_count":26,"area":null},{"id":"2101.05471","title":"Towards Practical Adam: Non-Convexity, Convergence Theory, and Mini-Batch Acceleration","created":"2021-01-14","cited_by_count":25,"area":null},{"id":"2102.05320","title":"An Adaptive Stochastic Sequential Quadratic Programming with Differentiable Exact Augmented Lagrangians","created":"2021-02-10","cited_by_count":25,"area":null},{"id":"2105.00872","title":"Convergence Analysis and System Design for Federated Learning over Wireless Networks","created":"2021-04-30","cited_by_count":25,"area":null},{"id":"2106.02588","title":"Stochastic gradient descent with noise of machine learning type. Part II: Continuous time analysis","created":"2021-06-04","cited_by_count":25,"area":null},{"id":"2206.07082","title":"Stability and Generalization of Stochastic Optimization with Nonconvex and Nonsmooth Problems","created":"2022-06-14","cited_by_count":25,"area":null}],"next_cursor":"100"}