{"total":214,"rows":[{"id":"2110.06296","title":"The Role of Permutation Invariance in Linear Mode Connectivity of Neural Networks","created":"2021-10-12","cited_by_count":291,"area":null},{"id":"2105.14368","title":"Fit without fear: remarkable mathematical phenomena of deep learning through the prism of interpolation","created":"2021-05-29","cited_by_count":132,"area":null},{"id":"2211.08403","title":"REPAIR: REnormalizing Permuted Activations for Interpolation Repair","created":"2022-11-15","cited_by_count":120,"area":null},{"id":"2104.11375","title":"Decentralized Federated Averaging","created":"2021-04-23","cited_by_count":114,"area":null},{"id":"2402.01868","title":"Challenges in Training PINNs: A Loss Landscape Perspective","created":"2024-02-02","cited_by_count":113,"area":null},{"id":"2209.08709","title":"BOME! Bilevel Optimization Made Easy: A Simple First-Order Approach","created":"2022-09-19","cited_by_count":105,"area":null},{"id":"2306.02235","title":"Learning Linear Causal Representations from Interventions under General Nonlinear Mixing","created":"2023-06-04","cited_by_count":92,"area":null},{"id":"2303.09105","title":"Rethinking Model Ensemble in Transfer-based Adversarial Attacks","created":"2023-03-16","cited_by_count":89,"area":null},{"id":"2302.05185","title":"On Penalty-based Bilevel Gradient Descent Method","created":"2023-02-10","cited_by_count":87,"area":null},{"id":"2302.14017","title":"Full Stack Optimization of Transformer Inference: a Survey","created":"2023-02-27","cited_by_count":80,"area":null},{"id":"2505.01618","title":"Don't be lazy: CompleteP enables compute-efficient deep transformers","created":"2025-05-02","cited_by_count":78,"area":null},{"id":"2206.00939","title":"Gradient flow dynamics of shallow ReLU networks for square loss and orthogonal inputs","created":"2022-06-02","cited_by_count":76,"area":null},{"id":"2505.13416","title":"Gluon: Making Muon & Scion Great Again! (Bridging Theory and Practice of LMO-based Optimizers for LLMs)","created":"2025-05-19","cited_by_count":72,"area":null},{"id":"2112.07628","title":"Training Multi-Layer Over-Parametrized Neural Network in Subquadratic Time","created":"2021-12-14","cited_by_count":62,"area":null},{"id":"2309.01753","title":"On Penalty Methods for Nonconvex Bilevel Optimization and First-Order Stochastic Approximation","created":"2023-09-04","cited_by_count":51,"area":null},{"id":"2209.09211","title":"Neural Collapse with Normalized Features: A Geometric Analysis over the Riemannian Manifold","created":"2022-09-19","cited_by_count":50,"area":null},{"id":"2112.05604","title":"Faster Single-loop Algorithms for Minimax Optimization without Strong Concavity","created":"2021-12-10","cited_by_count":49,"area":null},{"id":"2301.00712","title":"On Finding Small Hyper-Gradients in Bilevel Optimization: Hardness Results and Improved Analysis","created":"2023-01-02","cited_by_count":39,"area":null},{"id":"2203.04850","title":"Federated Minimax Optimization: Improved Convergence Analyses and Algorithms","created":"2022-03-09","cited_by_count":37,"area":null},{"id":"2305.15027","title":"A Rigorous Link between Deep Ensembles and (Variational) Bayesian Methods","created":"2023-05-24","cited_by_count":32,"area":null},{"id":"2302.03109","title":"On the Convergence of Federated Averaging with Cyclic Client Participation","created":"2023-02-06","cited_by_count":31,"area":null},{"id":"2306.04815","title":"Catapults in SGD: spikes in the training loss and their impact on generalization through feature learning","created":"2023-06-07","cited_by_count":31,"area":null},{"id":"2402.05738","title":"Implicit Bias and Fast Convergence Rates for Self-attention","created":"2024-02-08","cited_by_count":31,"area":null},{"id":"2211.06506","title":"Spectral Evolution and Invariance in Linear-width Neural Networks","created":"2022-11-11","cited_by_count":30,"area":null},{"id":"2107.12723","title":"Stability & Generalisation of Gradient Descent for Shallow Neural Networks without the Neural Tangent Kernel","created":"2021-07-27","cited_by_count":26,"area":null},{"id":"2111.01875","title":"Subquadratic Overparameterization for Shallow Neural Networks","created":"2021-11-02","cited_by_count":26,"area":null},{"id":"2203.06735","title":"Private Non-Convex Federated Learning Without a Trusted Server","created":"2022-03-13","cited_by_count":26,"area":null},{"id":"2203.16462","title":"Convergence of gradient descent for deep neural networks","created":"2022-03-30","cited_by_count":24,"area":null},{"id":"2211.02254","title":"How Does Adaptive Optimization Impact Local Neural Network Geometry?","created":"2022-11-04","cited_by_count":24,"area":null},{"id":"2305.13404","title":"Improving Convergence and Generalization Using Parameter Symmetries","created":"2023-05-22","cited_by_count":23,"area":null},{"id":"2201.09418","title":"Approximation bounds for norm constrained neural networks with applications to regression and GANs","created":"2022-01-24","cited_by_count":22,"area":null},{"id":"2306.02601","title":"Aiming towards the minimizers: fast convergence of SGD for overparametrized problems","created":"2023-06-05","cited_by_count":22,"area":null},{"id":"2106.13792","title":"Proxy Convexity: A Unified Framework for the Analysis of Neural Networks Trained by Gradient Descent","created":"2021-06-25","cited_by_count":21,"area":null},{"id":"2209.04036","title":"Functional dimension of feedforward ReLU neural networks","created":"2022-09-08","cited_by_count":21,"area":null},{"id":"2212.14457","title":"Bayesian Interpolation with Deep Linear Networks","created":"2022-12-29","cited_by_count":21,"area":null},{"id":"2302.09235","title":"Generalization and Stability of Interpolating Neural Networks with Minimal Width","created":"2023-02-18","cited_by_count":21,"area":null},{"id":"2207.00391","title":"A Theoretical Analysis of the Learning Dynamics under Class Imbalance","created":"2022-07-01","cited_by_count":20,"area":null},{"id":"2309.01213","title":"Implicit regularization of deep residual networks towards neural ODEs","created":"2023-09-03","cited_by_count":20,"area":null},{"id":"2403.20233","title":"Functional Bilevel Optimization for Machine Learning","created":"2024-03-29","cited_by_count":20,"area":null},{"id":"2204.12446","title":"Beyond Lipschitz: Sharp Generalization and Excess Risk Bounds for Full-Batch GD","created":"2022-04-26","cited_by_count":19,"area":null},{"id":"2311.14646","title":"More is Better in Modern Machine Learning: when Infinite Overparameterization is Optimal and Overfitting is Obligatory","created":"2023-11-24","cited_by_count":18,"area":null},{"id":"2111.06063","title":"On the Equivalence between Neural Network and Support Vector Machine","created":"2021-11-11","cited_by_count":17,"area":null},{"id":"2101.04968","title":"Learning with Gradient Descent and Weakly Convex Losses","created":"2021-01-13","cited_by_count":16,"area":null},{"id":"2403.00986","title":"Merging Text Transformer Models from Different Initializations","created":"2024-03-01","cited_by_count":16,"area":null},{"id":"2306.12747","title":"Don't be so Monotone: Relaxing Stochastic Line Search in Over-Parameterized Models","created":"2023-06-22","cited_by_count":15,"area":null},{"id":"2304.05187","title":"Automatic Gradient Descent: Deep Learning without Hyperparameters","created":"2023-04-11","cited_by_count":14,"area":null},{"id":"2307.11782","title":"Convergence of Adam for Non-convex Objectives: Relaxed Hyperparameters and Non-ergodic Case","created":"2023-07-20","cited_by_count":14,"area":null},{"id":"2407.20999","title":"MoFO: Momentum-Filtered Optimizer for Mitigating Forgetting in LLM Fine-Tuning","created":"2024-07-30","cited_by_count":14,"area":null},{"id":"2505.03793","title":"LENSLLM: Unveiling Fine-Tuning Dynamics for LLM Selection","created":"2025-05-01","cited_by_count":14,"area":null},{"id":"2204.10782","title":"On Feature Learning in Neural Networks with Global Convergence Guarantees","created":"2022-04-22","cited_by_count":13,"area":null},{"id":"2311.01575","title":"On the Convergence of Encoder-only Shallow Transformers","created":"2023-11-02","cited_by_count":13,"area":null},{"id":"2405.14630","title":"Bounds for the smallest eigenvalue of the NTK for arbitrary spherical data of arbitrary dimension","created":"2024-05-23","cited_by_count":13,"area":null},{"id":"2502.07923","title":"Sign Operator for Coping with Heavy-Tailed Noise in Non-Convex Optimization: High Probability Bounds Under $(L_0, L_1)$-Smoothness","created":"2025-02-11","cited_by_count":13,"area":null},{"id":"2505.17646","title":"Unveiling the Basin-Like Loss Landscape in Large Language Models","created":"2025-05-23","cited_by_count":13,"area":null},{"id":"2505.20172","title":"A Theoretical Framework for Grokking: Interpolation followed by Riemannian Norm Minimisation","created":"2025-05-26","cited_by_count":13,"area":null},{"id":"2204.14126","title":"Wide and Deep Neural Networks Achieve Optimality for Classification","created":"2022-04-29","cited_by_count":12,"area":null},{"id":"2310.08287","title":"A Symmetry-Aware Exploration of Bayesian Neural Network Posteriors","created":"2023-10-12","cited_by_count":12,"area":null},{"id":"2406.06838","title":"Stable Minima Cannot Overfit in Univariate ReLU Networks: Generalization by Large Step Sizes","created":"2024-06-10","cited_by_count":12,"area":null},{"id":"2206.11872","title":"Provable Acceleration of Heavy Ball beyond Quadratics for a Class of Polyak-\\L{}ojasiewicz Functions when the Non-Convexity is Averaged-Out","created":"2022-06-22","cited_by_count":11,"area":null},{"id":"2211.13668","title":"Zeroth-Order Alternating Gradient Descent Ascent Algorithms for a Class of Nonconvex-Nonconcave Minimax Problems","created":"2022-11-24","cited_by_count":11,"area":null},{"id":"2502.02870","title":"Uncertainty Quantification with the Empirical Neural Tangent Kernel","created":"2025-02-05","cited_by_count":11,"area":null},{"id":"2302.05823","title":"Data efficiency and extrapolation trends in neural network interatomic potentials","created":"2023-02-12","cited_by_count":10,"area":null},{"id":"2402.11173","title":"How to Make the Gradients Small Privately: Improved Rates for Differentially Private Non-Convex Optimization","created":"2024-02-17","cited_by_count":10,"area":null},{"id":"2208.07951","title":"On the generalization of learning algorithms that do not converge","created":"2022-08-16","cited_by_count":9,"area":null},{"id":"2305.07810","title":"Depth Dependence of $\\mu$P Learning Rates in ReLU MLPs","created":"2023-05-13","cited_by_count":9,"area":null},{"id":"2201.06656","title":"Generalization in Supervised Learning Through Riemannian Contraction","created":"2022-01-17","cited_by_count":8,"area":null},{"id":"2209.13262","title":"Exploring the Algorithm-Dependent Generalization of AUPRC Optimization with List Stability","created":"2022-09-27","cited_by_count":8,"area":null},{"id":"2410.12455","title":"Loss Landscape Characterization of Neural Networks without Over-Parametrization","created":"2024-10-16","cited_by_count":8,"area":null},{"id":"2502.07209","title":"Enhancing Physics-Informed Neural Networks Through Feature Engineering","created":"2025-02-11","cited_by_count":8,"area":null},{"id":"2209.15106","title":"Restricted Strong Convexity of Deep Learning Models with Smooth Activations","created":"2022-09-29","cited_by_count":7,"area":null},{"id":"2210.05371","title":"On skip connections and normalisation layers in deep optimisation","created":"2022-10-10","cited_by_count":7,"area":null},{"id":"2303.03027","title":"Critical Points and Convergence Analysis of Generative Deep Linear Networks Trained with Bures-Wasserstein Loss","created":"2023-03-06","cited_by_count":7,"area":null},{"id":"2306.11922","title":"No Wrong Turns: The Simple Geometry Of Neural Networks Optimization Paths","created":"2023-06-20","cited_by_count":7,"area":null},{"id":"2403.15654","title":"The Effectiveness of Local Updates for Decentralized Learning under Data Heterogeneity","created":"2024-03-23","cited_by_count":7,"area":null},{"id":"2410.10024","title":"Sharper Guarantees for Learning Neural Network Classifiers with Gradient Methods","created":"2024-10-13","cited_by_count":7,"area":null},{"id":"2508.16817","title":"Predictability Enables Parallelization of Nonlinear State Space Models","created":"2025-08-22","cited_by_count":7,"area":null},{"id":"2603.21191","title":"On the Role of Batch Size in Stochastic Conditional Gradient Methods","created":"2026-03-22","cited_by_count":7,"area":null},{"id":"2203.05104","title":"Transition to Linearity of Wide Neural Networks is an Emerging Property of Assembling Weak Models","created":"2022-03-10","cited_by_count":6,"area":null},{"id":"2210.13983","title":"Optimization for Amortized Inverse Problems","created":"2022-10-25","cited_by_count":6,"area":null},{"id":"2211.03241","title":"Neural PDE Solvers for Irregular Domains","created":"2022-11-07","cited_by_count":6,"area":null},{"id":"2302.02904","title":"Rethinking Gauss-Newton for learning over-parameterized models","created":"2023-02-06","cited_by_count":6,"area":null},{"id":"2305.08813","title":"Better NTK Conditioning: A Free Lunch from (ReLU) Nonlinear Activation in Wide Neural Networks","created":"2023-05-15","cited_by_count":6,"area":null},{"id":"2403.18136","title":"Identifying Backdoored Graphs in Graph Neural Network Training: An Explanation-Based Approach with Novel Metrics","created":"2024-03-26","cited_by_count":6,"area":null},{"id":"2201.04545","title":"On generalization bounds for deep networks based on loss surface implicit regularization","created":"2022-01-12","cited_by_count":5,"area":null},{"id":"2209.00771","title":"Optimizing the Performative Risk under Weak Convexity Assumptions","created":"2022-09-02","cited_by_count":5,"area":null},{"id":"2210.16400","title":"Flatter, faster: scaling momentum for optimal speedup of SGD","created":"2022-10-28","cited_by_count":5,"area":null},{"id":"2301.08117","title":"Convergence beyond the over-parameterized regime using Rayleigh quotients","created":"2023-01-19","cited_by_count":5,"area":null},{"id":"2306.02422","title":"A Generalized Alternating Method for Bilevel Learning under the Polyak-{\\L}ojasiewicz Condition","created":"2023-06-04","cited_by_count":5,"area":null},{"id":"2307.15868","title":"Faster Stochastic Algorithms for Minimax Optimization under Polyak--{\\L}ojasiewicz Conditions","created":"2023-07-29","cited_by_count":5,"area":null},{"id":"2311.15365","title":"A convergence result of a continuous model of deep learning via a \\L{}ojasiewicz--Simon inequality","created":"2023-11-26","cited_by_count":5,"area":null},{"id":"2401.06980","title":"Joint Unsupervised and Supervised Training for Automatic Speech Recognition via Bilevel Optimization","created":"2024-01-13","cited_by_count":5,"area":null},{"id":"2410.09383","title":"Deep Transfer Learning: Model Framework and Error Analysis","created":"2024-10-12","cited_by_count":5,"area":null},{"id":"2504.01898","title":"Analysis of an Idealized Stochastic Polyak Method and its Application to Black-Box Model Distillation","created":"2025-04-02","cited_by_count":5,"area":null},{"id":"2605.13079","title":"Spectral Flattening Is All Muon Needs: How Orthogonalization Controls Learning Rate and Convergence","created":"2026-05-13","cited_by_count":5,"area":null},{"id":"2108.00259","title":"How much pre-training is enough to discover a good subnetwork?","created":"2021-07-31","cited_by_count":4,"area":null},{"id":"2205.11786","title":"Transition to Linearity of General Neural Networks with Directed Acyclic Graph Architecture","created":"2022-05-24","cited_by_count":4,"area":null},{"id":"2206.14862","title":"Momentum Diminishes the Effect of Spectral Bias in Physics-Informed Neural Networks","created":"2022-06-29","cited_by_count":4,"area":null},{"id":"2301.09511","title":"On the Convergence of the Gradient Descent Method with Stochastic Fixed-point Rounding Errors under the Polyak-Lojasiewicz Inequality","created":"2023-01-23","cited_by_count":4,"area":null},{"id":"2304.10552","title":"Approximation and interpolation of deep neural networks","created":"2023-04-20","cited_by_count":4,"area":null},{"id":"2305.16242","title":"Two-timescale Extragradient for Finding Local Minimax Points","created":"2023-05-25","cited_by_count":4,"area":null}],"next_cursor":"100"}