{"list":"other","total":79,"rows":[{"id":"1907.11692","title":"RoBERTa: A Robustly Optimized BERT Pretraining Approach","created":"2019-07-26","cited_by_count":13372},{"id":"2010.01412","title":"Sharpness-Aware Minimization for Efficiently Improving Generalization","created":"2020-10-03","cited_by_count":1417},{"id":"1609.04836","title":"On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima","created":"2016-09-15","cited_by_count":1352},{"id":"1803.05407","title":"Averaging Weights Leads to Wider Optima and Better Generalization","created":"2018-03-14","cited_by_count":1209},{"id":"2202.10054","title":"Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution","created":"2022-02-21","cited_by_count":739},{"id":"1309.5549","title":"Stochastic First- and Zeroth-order Methods for Nonconvex Stochastic Programming","created":"2013-09-22","cited_by_count":724},{"id":"1912.00049","title":"Square Attack: a query-efficient black-box adversarial attack via random search","created":"2019-11-29","cited_by_count":685},{"id":"1708.03999","title":"ZOO: Zeroth Order Optimization based Black-box Attacks to Deep Neural Networks without Training Substitute Models","created":"2017-08-14","cited_by_count":666},{"id":"1706.08947","title":"Exploring Generalization in Deep Learning","created":"2017-06-27","cited_by_count":600},{"id":"1703.03864","title":"Evolution Strategies as a Scalable Alternative to Reinforcement Learning","created":"2017-03-10","cited_by_count":574},{"id":"1703.11008","title":"Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Data","created":"2017-03-31","cited_by_count":533},{"id":"1703.04933","title":"Sharp Minima Can Generalize For Deep Nets","created":"2017-03-15","cited_by_count":466},{"id":"2103.00065","title":"Gradient Descent on Neural Networks Typically Occurs at the Edge of Stability","created":"2021-02-26","cited_by_count":427},{"id":"1611.01838","title":"Entropy-SGD: Biasing Gradient Descent Into Wide Valleys","created":"2016-11-06","cited_by_count":359},{"id":"2305.17333","title":"Fine-Tuning Language Models with Just Forward Passes","created":"2023-05-27","cited_by_count":345},{"id":"2102.11600","title":"ASAM: Adaptive Sharpness-Aware Minimization for Scale-Invariant Learning of Deep Neural Networks","created":"2021-02-23","cited_by_count":293},{"id":"1807.01695","title":"SPIDER: Near-Optimal Non-Convex Optimization via Stochastic Path Integrated Differential Estimator","created":"2018-07-04","cited_by_count":287},{"id":"1312.2139","title":"Optimal rates for zero-order convex optimization: the power of two function evaluations","created":"2013-12-07","cited_by_count":198},{"id":"2203.08065","title":"Surrogate Gap Minimization Improves Sharpness-Aware Training","created":"2022-03-15","cited_by_count":172},{"id":"2209.15594","title":"Self-Stabilization: The Implicit Bias of Gradient Descent at the Edge of Stability","created":"2022-09-30","cited_by_count":155},{"id":"2206.06232","title":"Towards Understanding Sharpness-Aware Minimization","created":"2022-06-13","cited_by_count":146},{"id":"2203.02714","title":"Towards Efficient and Scalable Sharpness-Aware Minimization","created":"2022-03-05","cited_by_count":129},{"id":"2205.09745","title":"Understanding Gradient Descent on Edge of Stability in Deep Learning","created":"2022-05-19","cited_by_count":128},{"id":"1904.09080","title":"Implicit regularization for deep neural networks driven by an Ornstein-Uhlenbeck like process","created":"2019-04-19","cited_by_count":123},{"id":"2106.06530","title":"Label Noise SGD Provably Prefers Flat Global Minimizers","created":"2021-06-11","cited_by_count":121},{"id":"2402.11592","title":"Revisiting Zeroth-Order Optimization for Memory-Efficient LLM Fine-Tuning: A Benchmark","created":"2024-02-18","cited_by_count":114},{"id":"1507.08752","title":"An Optimal Algorithm for Bandit and Zero-Order Convex Optimization with Two-Point Feedback","created":"2015-07-31","cited_by_count":111},{"id":"1805.10367","title":"Zeroth-Order Stochastic Variance Reduction for Nonconvex Optimization","created":"2018-05-25","cited_by_count":111},{"id":"2110.06914","title":"What Happens after SGD Reaches Zero Loss? --A Mathematical Framework","created":"2021-10-13","cited_by_count":107},{"id":"2205.14083","title":"Sharpness-Aware Training for Free","created":"2022-05-27","cited_by_count":95},{"id":"1902.00744","title":"Asymmetric Valleys: Beyond Sharp and Flat Local Minima","created":"2019-02-02","cited_by_count":90},{"id":"2206.07085","title":"Understanding the Generalization Benefit of Normalization Layers: Sharpness Reduction","created":"2022-06-14","cited_by_count":86},{"id":"2210.14199","title":"Same Pre-training Loss, Better Downstream: Implicit Bias Matters for Language Models","created":"2022-10-25","cited_by_count":84},{"id":"2010.04925","title":"Regularizing Neural Networks via Adversarial Model Perturbation","created":"2020-10-10","cited_by_count":78},{"id":"2202.00661","title":"When Do Flat Minima Optimizers Work?","created":"2022-02-01","cited_by_count":78},{"id":"1910.06513","title":"ZO-AdaMM: Zeroth-Order Adaptive Momentum Method for Black-Box Optimization","created":"2019-10-15","cited_by_count":76},{"id":"2207.12678","title":"Analyzing Sharpness along GD Trajectory: Progressive Sharpening and Edge of Stability","created":"2022-07-26","cited_by_count":63},{"id":"2201.09531","title":"Communication-Efficient Stochastic Zeroth-Order Optimization for Federated Learning","created":"2022-01-24","cited_by_count":59},{"id":"2303.03108","title":"Gradient Norm Aware Minimization Seeks First-Order Flatness and Improves Generalization","created":"2023-03-03","cited_by_count":57},{"id":"2312.06353","title":"Federated Full-Parameter Tuning of Billion-Sized Language Models with Communication Cost under 18 Kilobytes","created":"2023-12-11","cited_by_count":55},{"id":"2402.15751","title":"Sparse MeZO: Less Parameters for Better Performance in Zeroth-Order LLM Fine-Tuning","created":"2024-02-24","cited_by_count":54},{"id":"2404.08080","title":"Variance-reduced Zeroth-Order Methods for Fine-Tuning Language Models","created":"2024-04-11","cited_by_count":54},{"id":"2402.15173","title":"Second-Order Fine-Tuning without Pain for LLMs:A Hessian Informed Zeroth-Order Optimizer","created":"2024-02-23","cited_by_count":53},{"id":"2210.01513","title":"The Dynamics of Sharpness-Aware Minimization: Bouncing Across Ravines and Drifting Towards Wide Minima","created":"2022-10-04","cited_by_count":51},{"id":"2212.07469","title":"Learning threshold neurons via the \"edge of stability\"","created":"2022-12-14","cited_by_count":51},{"id":"2410.07698","title":"Enhancing Zeroth-order Fine-tuning for Language Models with Low-rank Structures","created":"2024-10-10","cited_by_count":50},{"id":"1910.12166","title":"Improved Zeroth-Order Variance Reduced Algorithms and Analysis for Nonconvex Optimization","created":"2019-10-27","cited_by_count":47},{"id":"1606.00498","title":"A Comprehensive Linear Speedup Analysis for Asynchronous Stochastic Parallel Optimization from Zeroth-Order to First-Order","created":"2016-06-01","cited_by_count":44},{"id":"2209.05045","title":"Gradient-Free Methods for Deterministic and Stochastic Nonsmooth Nonconvex Optimization","created":"2022-09-12","cited_by_count":43},{"id":"2307.11007","title":"Sharpness Minimization Algorithms Do Not Only Minimize Sharpness To Achieve Better Generalization","created":"2023-07-20","cited_by_count":39},{"id":"1908.02734","title":"Stochastic First-order Methods for Convex and Nonconvex Functional Constrained Optimization","created":"2019-08-07","cited_by_count":38},{"id":"1809.06474","title":"Zeroth-order Nonconvex Stochastic Optimization: Handling Constraints, High-Dimensionality and Saddle-Points","created":"2018-09-17","cited_by_count":37},{"id":"2309.15639","title":"Enhancing Sharpness-Aware Optimization Through Variance Suppression","created":"2023-09-27","cited_by_count":37},{"id":"2203.03756","title":"Flat minima generalize for low-rank matrix recovery","created":"2022-03-07","cited_by_count":28},{"id":"2406.02913","title":"Zeroth-Order Fine-Tuning of LLMs with Extreme Sparsity","created":"2024-06-05","cited_by_count":28},{"id":"2310.09639","title":"DPZero: Private Fine-Tuning of Language Models without Backpropagation","created":"2023-10-14","cited_by_count":26},{"id":"2312.15184","title":"ZO-AdaMU Optimizer: Adapting Perturbation by the Momentum and Uncertainty in Zeroth-order Optimization","created":"2023-12-23","cited_by_count":25},{"id":"1605.08165","title":"Highly-Smooth Zero-th Order Online Optimization Vianney Perchet","created":"2016-05-26","cited_by_count":23},{"id":"2307.04504","title":"An Algorithm with Optimal Dimension-Dependence for Zero-Order Nonsmooth Nonconvex Stochastic Optimization","created":"2023-07-10","cited_by_count":23},{"id":"2001.07819","title":"Zeroth-Order Algorithms for Nonconvex Minimax Problems with Improved Complexities","created":"2020-01-22","cited_by_count":21},{"id":"2301.06428","title":"Faster Gradient-Free Algorithms for Nonsmooth Nonconvex Stochastic Optimization","created":"2023-01-16","cited_by_count":17},{"id":"2102.04297","title":"Eliminating Sharp Minima from SGD with Truncated Heavy-tailed Noise","created":"2021-02-08","cited_by_count":14},{"id":"2306.08553","title":"Noise Stability Optimization for Finding Flat Minima: A Hessian-based Regularization Approach","created":"2023-06-14","cited_by_count":14},{"id":"2305.15659","title":"How to escape sharp minima with random perturbations","created":"2023-05-25","cited_by_count":13},{"id":"2306.02159","title":"Gradient-free optimization of highly smooth functions: improved analysis and a new algorithm","created":"2023-06-03","cited_by_count":12},{"id":"2307.05753","title":"Zeroth-order Optimization with Weak Dimension Dependency","created":"2023-07-11","cited_by_count":11},{"id":"2410.06441","title":"Addax: Utilizing Zeroth-Order Gradients to Improve Memory Efficiency and Performance of SGD for Fine-Tuning Language Models","created":"2024-10-09","cited_by_count":11},{"id":"1910.13021","title":"Efficiently avoiding saddle points with zero order methods: No gradients required","created":"2019-10-29","cited_by_count":10},{"id":"2410.14802","title":"Implicit Regularization of Sharpness-Aware Minimization for Scale-Invariant Problems","created":"2024-10-18","cited_by_count":10},{"id":"2309.14506","title":"Zeroth-order Riemannian Averaging Stochastic Approximation Algorithms","created":"2023-09-25","cited_by_count":7},{"id":"2210.01496","title":"Zeroth-Order Negative Curvature Finding: Escaping Saddle Points without Gradients","created":"2022-10-04","cited_by_count":6},{"id":"2404.00158","title":"Fully Zeroth-Order Bilevel Programming via Gaussian Smoothing","created":"2024-03-29","cited_by_count":6},{"id":"2404.05318","title":"Stochastic Online Optimization for Cyber-Physical and Robotic Systems","created":"2024-04-08","cited_by_count":6},{"id":"2110.13265","title":"On the Second-order Convergence Properties of Random Search Methods","created":"2021-10-25","cited_by_count":5},{"id":"2209.13555","title":"Escaping saddle points in zeroth-order optimization: the power of two-point estimators","created":"2022-09-27","cited_by_count":5},{"id":"2502.19571","title":"LORENZA: Enhancing Generalization in Low-Rank Gradient LLM Training via Efficient Zeroth-Order Adaptive SAM","created":"2025-02-26","cited_by_count":5},{"id":"2403.12859","title":"Primal Methods for Variational Inequality Problems with Functional Constraints","created":"2024-03-19","cited_by_count":3},{"id":"2410.12457","title":"Sharpness-Aware Black-Box Optimization","created":"2024-10-16","cited_by_count":2},{"id":"2404.04355","title":"Gray-Box Nonlinear Feedback Optimization","created":"2024-04-05","cited_by_count":1}],"next_cursor":null}