{"total":545,"rows":[{"id":"2102.06810","title":"Understanding self-supervised Learning Dynamics without Contrastive Pairs","created":"2021-02-12","cited_by_count":256,"area":null},{"id":"2204.11127","title":"U-NO: U-shaped Neural Operators","created":"2022-04-23","cited_by_count":230,"area":null},{"id":"2105.02375","title":"A Geometric Analysis of Neural Collapse with Unconstrained Features","created":"2021-05-06","cited_by_count":227,"area":null},{"id":"2106.15590","title":"The Values Encoded in Machine Learning Research","created":"2021-06-29","cited_by_count":172,"area":null},{"id":"2101.12699","title":"Exploring Deep Neural Networks via Layer-Peeled Model: Minority Collapse in Imbalanced Training","created":"2021-01-29","cited_by_count":166,"area":null},{"id":"2206.15144","title":"Neural Networks can Learn Representations with Gradient Descent","created":"2022-06-30","cited_by_count":161,"area":null},{"id":"2103.10427","title":"The Low-Rank Simplicity Bias in Deep Networks","created":"2021-03-18","cited_by_count":154,"area":null},{"id":"2109.00650","title":"Dash: Semi-Supervised Learning with Dynamic Thresholding","created":"2021-09-01","cited_by_count":148,"area":null},{"id":"2105.14368","title":"Fit without fear: remarkable mathematical phenomena of deep learning through the prism of interpolation","created":"2021-05-29","cited_by_count":132,"area":null},{"id":"2112.08304","title":"On the Convergence and Robustness of Adversarial Training","created":"2021-12-15","cited_by_count":130,"area":null},{"id":"2105.12221","title":"Geometry of the Loss Landscape in Overparameterized Neural Networks: Symmetries and Invariances","created":"2021-05-25","cited_by_count":127,"area":null},{"id":"2106.06530","title":"Label Noise SGD Provably Prefers Flat Global Minimizers","created":"2021-06-11","cited_by_count":121,"area":null},{"id":"2210.05643","title":"A Kernel-Based View of Language Model Fine-Tuning","created":"2022-10-11","cited_by_count":121,"area":null},{"id":"2105.15134","title":"Toward Understanding the Feature Learning Process of Self-supervised Contrastive Learning","created":"2021-05-31","cited_by_count":119,"area":null},{"id":"2101.10588","title":"Generalization error of random features and kernel methods: hypercontractivity and kernel matrix concentration","created":"2021-01-26","cited_by_count":110,"area":null},{"id":"2102.02887","title":"Do We Actually Need Dense Over-Parameterization? In-Time Over-Parameterization in Sparse Training","created":"2021-02-04","cited_by_count":110,"area":null},{"id":"2110.06914","title":"What Happens after SGD Reaches Zero Loss? --A Mathematical Framework","created":"2021-10-13","cited_by_count":107,"area":null},{"id":"2205.09653","title":"Self-Consistent Dynamical Field Theory of Kernel Evolution in Wide Neural Networks","created":"2022-05-19","cited_by_count":105,"area":null},{"id":"2305.16380","title":"Scan and Snap: Understanding Training Dynamics and Token Composition in 1-layer Transformer","created":"2023-05-25","cited_by_count":102,"area":null},{"id":"2109.11676","title":"Theory of overparametrization in quantum neural networks","created":"2021-09-23","cited_by_count":100,"area":null},{"id":"2202.06526","title":"Benign Overfitting in Two-layer Convolutional Neural Networks","created":"2022-02-14","cited_by_count":91,"area":null},{"id":"2102.13219","title":"Learning with invariances in random features and kernel models","created":"2021-02-25","cited_by_count":89,"area":null},{"id":"2110.01654","title":"Improved architectures and training algorithms for deep operator networks","created":"2021-10-04","cited_by_count":86,"area":null},{"id":"2110.13905","title":"Gradient Descent on Two-layer Nets: Margin Maximization and Simplicity Bias","created":"2021-10-26","cited_by_count":83,"area":null},{"id":"2311.18817","title":"Dichotomy of Early and Late Phase Implicit Biases Can Provably Induce Grokking","created":"2023-11-30","cited_by_count":80,"area":null},{"id":"2106.15013","title":"Small random initialization is akin to spectral learning: Optimization and generalization guarantees for overparameterized low-rank matrix reconstruction","created":"2021-06-28","cited_by_count":78,"area":null},{"id":"2312.14688","title":"A Mathematical Guide to Operator Learning","created":"2023-12-22","cited_by_count":76,"area":null},{"id":"2202.08832","title":"Universality of empirical risk minimization","created":"2022-02-17","cited_by_count":75,"area":null},{"id":"2205.07344","title":"Policy Gradient Method For Robust Reinforcement Learning","created":"2022-05-15","cited_by_count":69,"area":null},{"id":"2105.11144","title":"Improved OOD Generalization via Adversarial Training and Pre-training","created":"2021-05-24","cited_by_count":66,"area":null},{"id":"2110.11526","title":"Wide Neural Networks Forget Less Catastrophically","created":"2021-10-21","cited_by_count":64,"area":null},{"id":"2109.14119","title":"Stochastic Training is Not Necessary for Generalization","created":"2021-09-29","cited_by_count":63,"area":null},{"id":"2207.12678","title":"Analyzing Sharpness along GD Trajectory: Progressive Sharpening and Edge of Stability","created":"2022-07-26","cited_by_count":63,"area":null},{"id":"2112.07628","title":"Training Multi-Layer Over-Parametrized Neural Network in Subquadratic Time","created":"2021-12-14","cited_by_count":62,"area":null},{"id":"2204.07697","title":"Theory of Graph Neural Networks: Representation and Learning","created":"2022-04-16","cited_by_count":61,"area":null},{"id":"2110.03237","title":"Score-based Generative Neural Networks for Large-Scale Optimal Transport","created":"2021-10-07","cited_by_count":60,"area":null},{"id":"2301.11300","title":"ZiCo: Zero-shot NAS via Inverse Coefficient of Variation on Gradients","created":"2023-01-26","cited_by_count":58,"area":null},{"id":"2306.13596","title":"Max-Margin Token Selection in Attention Mechanism","created":"2023-06-23","cited_by_count":58,"area":null},{"id":"2105.04550","title":"Optimization of Graph Neural Networks: Implicit Acceleration by Skip Connections and More Depth","created":"2021-05-10","cited_by_count":55,"area":null},{"id":"2106.01606","title":"Exploring Memorization in Adversarial Training","created":"2021-06-03","cited_by_count":55,"area":null},{"id":"2310.00535","title":"JoMA: Demystifying Multilayer Transformers via JOint Dynamics of MLP and Attention","created":"2023-10-01","cited_by_count":52,"area":null},{"id":"2106.09017","title":"Bridging Multi-Task Learning and Meta-Learning: Towards Efficient Training and Effective Adaptation","created":"2021-06-16","cited_by_count":50,"area":null},{"id":"2202.05834","title":"Predicting Out-of-Distribution Error with the Projection Norm","created":"2022-02-11","cited_by_count":50,"area":null},{"id":"2211.08422","title":"Mechanistic Mode Connectivity","created":"2022-11-15","cited_by_count":50,"area":null},{"id":"2105.09513","title":"Deep Kronecker neural networks: A general framework for neural networks with adaptive activation functions","created":"2021-05-20","cited_by_count":49,"area":null},{"id":"2106.06770","title":"What can linearized neural networks actually say about generalization?","created":"2021-06-12","cited_by_count":49,"area":null},{"id":"2108.11371","title":"Understanding the Generalization of Adam in Learning Neural Networks with Proper Regularization","created":"2021-08-25","cited_by_count":49,"area":null},{"id":"2112.05604","title":"Faster Single-loop Algorithms for Minimax Optimization without Strong Concavity","created":"2021-12-10","cited_by_count":49,"area":null},{"id":"2201.01212","title":"AutoBalance: Optimized Loss Functions for Imbalanced Data","created":"2022-01-04","cited_by_count":49,"area":null},{"id":"2409.14623","title":"From Lazy to Rich: Exact Learning Dynamics in Deep Linear Networks","created":"2024-09-22","cited_by_count":49,"area":null},{"id":"2402.11867","title":"LoRA Training in the NTK Regime has No Spurious Local Minima","created":"2024-02-19","cited_by_count":48,"area":null},{"id":"2102.02410","title":"A Local Convergence Theory for Mildly Over-Parameterized Two-Layer Neural Network","created":"2021-02-04","cited_by_count":47,"area":null},{"id":"2101.09612","title":"On the Proof of Global Convergence of Gradient Descent for Deep ReLU Networks with Linear Widths","created":"2021-01-24","cited_by_count":44,"area":null},{"id":"2110.03177","title":"EE-Net: Exploitation-Exploration Neural Networks in Contextual Bandits","created":"2021-10-07","cited_by_count":44,"area":null},{"id":"2107.14702","title":"Towards General Function Approximation in Zero-Sum Markov Games","created":"2021-07-30","cited_by_count":43,"area":null},{"id":"2203.09376","title":"Escaping from the Barren Plateau via Gaussian Initializations in Deep Variational Quantum Circuits","created":"2022-03-17","cited_by_count":43,"area":null},{"id":"2301.11500","title":"Understanding Incremental Learning of Gradient Descent: A Fine-grained Analysis of Matrix Sensing","created":"2023-01-27","cited_by_count":42,"area":null},{"id":"2406.06158","title":"Get rich quick: exact solutions reveal how unbalanced initializations promote rapid feature learning","created":"2024-06-10","cited_by_count":42,"area":null},{"id":"2103.01210","title":"Quantifying the Benefit of Using Differentiable Learning over Tangent Kernels","created":"2021-03-01","cited_by_count":41,"area":null},{"id":"2206.02768","title":"The Neural Covariance SDE: Shaped Infinite Depth-and-Width Networks at Initialization","created":"2022-06-06","cited_by_count":41,"area":null},{"id":"2107.03144","title":"Neural Contextual Bandits without Regret","created":"2021-07-07","cited_by_count":40,"area":null},{"id":"2307.11353","title":"What can a Single Attention Layer Learn? A Study Through the Random Features Lens","created":"2023-07-21","cited_by_count":40,"area":null},{"id":"2207.04036","title":"Implicit Bias of Gradient Descent on Reparametrized Models: On Equivalence to Mirror Descent","created":"2022-07-08","cited_by_count":39,"area":null},{"id":"2105.14573","title":"Embedding Principle of Loss Landscape of Deep Neural Networks","created":"2021-05-30","cited_by_count":38,"area":null},{"id":"2211.16327","title":"On the Power of Foundation Models","created":"2022-11-29","cited_by_count":37,"area":null},{"id":"2301.12832","title":"Deep networks for system identification: a Survey","created":"2023-01-30","cited_by_count":37,"area":null},{"id":"2310.12680","title":"On the Optimization and Generalization of Multi-head Attention","created":"2023-10-19","cited_by_count":37,"area":null},{"id":"2205.10217","title":"Memorization and Optimization in Deep Neural Networks with Minimum Over-parameterization","created":"2022-05-20","cited_by_count":36,"area":null},{"id":"2303.15438","title":"On the Stepwise Nature of Self-Supervised Learning","created":"2023-03-27","cited_by_count":36,"area":null},{"id":"2102.07346","title":"On the Theory of Implicit Deep Learning: Global Convergence with Implicit Layers","created":"2021-02-15","cited_by_count":35,"area":null},{"id":"2206.10012","title":"Limitations of the NTK for Understanding Generalization in Deep Learning","created":"2022-06-20","cited_by_count":35,"area":null},{"id":"2107.06608","title":"Continuous vs. Discrete Optimization of Deep Neural Networks","created":"2021-07-14","cited_by_count":34,"area":null},{"id":"2107.11228","title":"Taxonomizing local versus global structure in neural network loss landscapes","created":"2021-07-23","cited_by_count":34,"area":null},{"id":"2306.03534","title":"Continual Learning in Linear Classification on Separable Data","created":"2023-06-06","cited_by_count":34,"area":null},{"id":"2106.04013","title":"The Future is Log-Gaussian: ResNets and Their Infinite-Depth-and-Width Limit at Initialization","created":"2021-06-07","cited_by_count":33,"area":null},{"id":"2111.13293","title":"KNAS: Green Neural Architecture Search","created":"2021-11-26","cited_by_count":33,"area":null},{"id":"2107.13741","title":"Self-Paced Contrastive Learning for Semi-supervised Medical Image Segmentation with Meta-labels","created":"2021-07-29","cited_by_count":32,"area":null},{"id":"2110.04622","title":"Does Preprocessing Help Training Over-parameterized Neural Networks?","created":"2021-10-09","cited_by_count":32,"area":null},{"id":"2305.02657","title":"On the Eigenvalue Decay Rates of a Class of Neural-Network Related Kernel Functions Defined on General Domains","created":"2023-05-04","cited_by_count":31,"area":null},{"id":"2306.04815","title":"Catapults in SGD: spikes in the training loss and their impact on generalization through feature learning","created":"2023-06-07","cited_by_count":31,"area":null},{"id":"2309.01020","title":"On the training and generalization of deep operator networks","created":"2023-09-02","cited_by_count":31,"area":null},{"id":"2210.00960","title":"Stability Analysis and Generalization Bounds of Adversarial Training","created":"2022-10-03","cited_by_count":30,"area":null},{"id":"2211.06506","title":"Spectral Evolution and Invariance in Linear-width Neural Networks","created":"2022-11-11","cited_by_count":30,"area":null},{"id":"2302.05933","title":"Generalization Ability of Wide Neural Networks on $\\mathbb{R}$","created":"2023-02-12","cited_by_count":30,"area":null},{"id":"2201.12293","title":"Understanding Why Generalized Reweighting Does Not Improve Over ERM","created":"2022-01-28","cited_by_count":29,"area":null},{"id":"2105.10585","title":"Properties of the After Kernel","created":"2021-05-21","cited_by_count":27,"area":null},{"id":"2205.14612","title":"Do Residual Neural Networks discretize Neural Ordinary Differential Equations?","created":"2022-05-29","cited_by_count":27,"area":null},{"id":"2212.03714","title":"Reconstructing Training Data from Model Gradient, Provably","created":"2022-12-07","cited_by_count":27,"area":null},{"id":"2203.10036","title":"On the Generalization Mystery in Deep Learning","created":"2022-03-18","cited_by_count":25,"area":null},{"id":"2302.10034","title":"Over-Parameterization Exponentially Slows Down Gradient Descent for Learning a Single Neuron","created":"2023-02-20","cited_by_count":25,"area":null},{"id":"2105.14417","title":"Overparameterization of deep ResNet: zero loss and mean-field analysis","created":"2021-05-30","cited_by_count":24,"area":null},{"id":"2203.16462","title":"Convergence of gradient descent for deep neural networks","created":"2022-03-30","cited_by_count":24,"area":null},{"id":"2306.08321","title":"Nonparametric regression using over-parameterized shallow ReLU neural networks","created":"2023-06-14","cited_by_count":24,"area":null},{"id":"2208.13614","title":"Neural Tangent Kernel: A Survey","created":"2022-08-29","cited_by_count":23,"area":null},{"id":"2106.03243","title":"Neural Active Learning with Performance Guarantees","created":"2021-06-06","cited_by_count":22,"area":null},{"id":"2106.03693","title":"Learning by Transference: Training Graph Neural Networks on Growing Graphs","created":"2021-06-07","cited_by_count":22,"area":null},{"id":"2110.02479","title":"Exponentially Many Local Minima in Quantum Neural Networks","created":"2021-10-06","cited_by_count":22,"area":null},{"id":"2111.02484","title":"Accelerated replica exchange stochastic gradient Langevin diffusion enhanced Bayesian DeepONet for solving noisy parametric PDEs","created":"2021-11-03","cited_by_count":22,"area":null},{"id":"2111.15527","title":"Embedding Principle: a hierarchical structure of loss landscape of deep neural networks","created":"2021-11-30","cited_by_count":22,"area":null},{"id":"2201.09418","title":"Approximation bounds for norm constrained neural networks with applications to regression and GANs","created":"2022-01-24","cited_by_count":22,"area":null}],"next_cursor":"100"}