{"total":625,"rows":[{"id":"2303.18223","title":"A Survey of Large Language Models","created":"2023-03-31","cited_by_count":2576,"area":null},{"id":"2209.11895","title":"In-context Learning and Induction Heads","created":"2022-09-24","cited_by_count":964,"area":null},{"id":"2201.02177","title":"Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets","created":"2022-01-06","cited_by_count":629,"area":null},{"id":"2305.16264","title":"Scaling Data-Constrained Language Models","created":"2023-05-25","cited_by_count":403,"area":null},{"id":"2102.06701","title":"Explaining Neural Scaling Laws","created":"2021-02-12","cited_by_count":363,"area":null},{"id":"2110.06296","title":"The Role of Permutation Invariance in Linear Mode Connectivity of Neural Networks","created":"2021-10-12","cited_by_count":291,"area":null},{"id":"2102.01293","title":"Scaling Laws for Transfer","created":"2021-02-02","cited_by_count":272,"area":null},{"id":"2205.10343","title":"Towards Understanding Grokking: An Effective Theory of Representation Learning","created":"2022-05-20","cited_by_count":230,"area":null},{"id":"2105.02375","title":"A Geometric Analysis of Neural Collapse with Unconstrained Features","created":"2021-05-06","cited_by_count":227,"area":null},{"id":"2205.10770","title":"Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Models","created":"2022-05-22","cited_by_count":220,"area":null},{"id":"2307.05695","title":"ReLoRA: High-Rank Training Through Low-Rank Updates","created":"2023-07-11","cited_by_count":204,"area":null},{"id":"2103.10427","title":"The Low-Rank Simplicity Bias in Deep Networks","created":"2021-03-18","cited_by_count":154,"area":null},{"id":"2205.10487","title":"Scaling Laws and Interpretability of Learning from Repeated Data","created":"2022-05-21","cited_by_count":152,"area":null},{"id":"2105.12806","title":"A Universal Law of Robustness via Isoperimetry","created":"2021-05-26","cited_by_count":151,"area":null},{"id":"2411.00816","title":"CycleResearcher: Improving Automated Research via Automated Review","created":"2024-10-28","cited_by_count":148,"area":null},{"id":"2206.06232","title":"Towards Understanding Sharpness-Aware Minimization","created":"2022-06-13","cited_by_count":146,"area":null},{"id":"2210.01117","title":"Omnigrok: Grokking Beyond Algorithmic Data","created":"2022-10-03","cited_by_count":146,"area":null},{"id":"2309.00667","title":"Taken out of context: On measuring situational awareness in LLMs","created":"2023-09-01","cited_by_count":143,"area":null},{"id":"2404.07965","title":"Rho-1: Not All Tokens Are What You Need","created":"2024-04-11","cited_by_count":133,"area":null},{"id":"2105.14368","title":"Fit without fear: remarkable mathematical phenomena of deep learning through the prism of interpolation","created":"2021-05-29","cited_by_count":132,"area":null},{"id":"2303.03329","title":"End-to-End Speech Recognition: A Survey","created":"2023-03-03","cited_by_count":129,"area":null},{"id":"2106.13799","title":"Assessing Generalization of SGD via Disagreement","created":"2021-06-25","cited_by_count":117,"area":null},{"id":"2108.12284","title":"The Devil is in the Detail: Simple Tricks Improve Systematic Generalization of Transformers","created":"2021-08-26","cited_by_count":117,"area":null},{"id":"2210.16859","title":"A Solvable Model of Neural Scaling Laws","created":"2022-10-30","cited_by_count":116,"area":null},{"id":"2106.15853","title":"Understanding and Improving Early Stopping for Learning with Noisy Labels","created":"2021-06-30","cited_by_count":113,"area":null},{"id":"2210.14891","title":"Broken Neural Scaling Laws","created":"2022-10-26","cited_by_count":109,"area":null},{"id":"2203.01238","title":"On the Optimization Landscape of Neural Collapse under MSE Loss: Global Optimality with Unconstrained Features","created":"2022-03-02","cited_by_count":97,"area":null},{"id":"2104.02133","title":"SpeechStew: Simply Mix All Available Speech Recognition Data to Train One Large Neural Network","created":"2021-04-05","cited_by_count":83,"area":null},{"id":"2212.09803","title":"Training Trajectories of Language Models Across Scales","created":"2022-12-19","cited_by_count":81,"area":null},{"id":"2112.00029","title":"Pixelated Butterfly: Simple and Efficient Sparse training for Neural Network Models","created":"2021-11-30","cited_by_count":76,"area":null},{"id":"2211.13609","title":"PAC-Bayes Compression Bounds So Tight That They Can Explain Generalization","created":"2022-11-24","cited_by_count":76,"area":null},{"id":"2103.01550","title":"Label-Imbalanced and Group-Sensitive Classification under Overparameterization","created":"2021-03-02","cited_by_count":74,"area":null},{"id":"2105.14602","title":"On the geometry of generalization and memorization in deep neural networks","created":"2021-05-30","cited_by_count":73,"area":null},{"id":"2310.04415","title":"Why Do We Need Weight Decay in Modern Deep Learning?","created":"2023-10-06","cited_by_count":72,"area":null},{"id":"2103.12692","title":"Benign Overfitting of Constant-Stepsize SGD for Linear Regression","created":"2021-03-23","cited_by_count":71,"area":null},{"id":"2103.10948","title":"The Shape of Learning Curves: a Review","created":"2021-03-19","cited_by_count":70,"area":null},{"id":"2210.05337","title":"SGD with Large Step Sizes Learns Sparse Features","created":"2022-10-11","cited_by_count":68,"area":null},{"id":"2103.05127","title":"Model Complexity of Deep Learning: A Survey","created":"2021-03-08","cited_by_count":65,"area":null},{"id":"2202.06985","title":"Deep Ensembles Work, But Are They Necessary?","created":"2022-02-14","cited_by_count":63,"area":null},{"id":"2103.03014","title":"Lost in Pruning: The Effects of Pruning Neural Networks beyond Test Accuracy","created":"2021-03-04","cited_by_count":62,"area":null},{"id":"2212.06347","title":"Reliable extrapolation of deep neural operators informed by physics or sparse observations","created":"2022-12-13","cited_by_count":62,"area":null},{"id":"2505.24832","title":"How much do language models memorize?","created":"2025-05-30","cited_by_count":61,"area":null},{"id":"2106.03216","title":"On Memorization in Probabilistic Deep Generative Models","created":"2021-06-06","cited_by_count":60,"area":null},{"id":"2107.08444","title":"A Theory of PAC Learnability of Partial Concept Classes","created":"2021-07-18","cited_by_count":60,"area":null},{"id":"2306.11987","title":"Training Transformers with 4-bit Integers","created":"2023-06-21","cited_by_count":60,"area":null},{"id":"2307.06290","title":"Instruction Mining: Instruction Data Selection for Tuning Large Language Models","created":"2023-07-12","cited_by_count":58,"area":null},{"id":"2109.02355","title":"A Farewell to the Bias-Variance Tradeoff? An Overview of the Theory of Overparameterized Machine Learning","created":"2021-09-06","cited_by_count":57,"area":null},{"id":"2203.08124","title":"Can Neural Nets Learn the Same Model Twice? Investigating Reproducibility and Double Descent from the Decision Boundary Perspective","created":"2022-03-15","cited_by_count":57,"area":null},{"id":"2503.02113","title":"Deep Learning is Not So Mysterious or Different","created":"2025-03-03","cited_by_count":56,"area":null},{"id":"2312.17173","title":"Non-Vacuous Generalization Bounds for Large Language Models","created":"2023-12-28","cited_by_count":53,"area":null},{"id":"2206.08675","title":"Understanding Robust Overfitting of Adversarial Training and Beyond","created":"2022-06-17","cited_by_count":51,"area":null},{"id":"2301.11526","title":"Direct Parameterization of Lipschitz-Bounded Deep Networks","created":"2023-01-27","cited_by_count":51,"area":null},{"id":"2303.06173","title":"Unifying Grokking and Double Descent","created":"2023-03-10","cited_by_count":50,"area":null},{"id":"2405.00592","title":"Scaling and renormalization in high-dimensional regression","created":"2024-05-01","cited_by_count":50,"area":null},{"id":"2110.00296","title":"Powerpropagation: A sparsity inducing weight reparameterisation","created":"2021-10-01","cited_by_count":49,"area":null},{"id":"2101.05544","title":"DICE: Diversity in Deep Ensembles via Conditional Redundancy Adversarial Estimation","created":"2021-01-14","cited_by_count":48,"area":null},{"id":"2307.14023","title":"Are Transformers with One Layer Self-Attention Using Low-Rank Weight Matrices Universal Approximators?","created":"2023-07-26","cited_by_count":47,"area":null},{"id":"2202.07105","title":"A Survey on Model Compression and Acceleration for Pretrained Language Models","created":"2022-02-15","cited_by_count":45,"area":null},{"id":"2209.15007","title":"Understanding Collapse in Non-Contrastive Siamese Representation Learning","created":"2022-09-29","cited_by_count":42,"area":null},{"id":"2305.18411","title":"Feature-Learning Networks Are Consistent Across Widths At Realistic Scales","created":"2023-05-28","cited_by_count":41,"area":null},{"id":"2405.20233","title":"Grokfast: Accelerated Grokking by Amplifying Slow Gradients","created":"2024-05-30","cited_by_count":41,"area":null},{"id":"2110.03187","title":"On the Optimal Memorization Power of ReLU Neural Networks","created":"2021-10-07","cited_by_count":40,"area":null},{"id":"2201.08025","title":"Low-Pass Filtering SGD for Recovering Flat Optima in the Deep Learning Optimization Landscape","created":"2022-01-20","cited_by_count":39,"area":null},{"id":"2209.13083","title":"Why neural networks find simple solutions: the many regularizers of geometric complexity","created":"2022-09-27","cited_by_count":39,"area":null},{"id":"2210.15031","title":"Characterizing Datapoints via Second-Split Forgetting","created":"2022-10-26","cited_by_count":37,"area":null},{"id":"2301.12832","title":"Deep networks for system identification: a Survey","created":"2023-01-30","cited_by_count":37,"area":null},{"id":"2303.14151","title":"Double Descent Demystified: Identifying, Interpreting & Ablating the Sources of a Deep Learning Puzzle","created":"2023-03-24","cited_by_count":37,"area":null},{"id":"2101.01761","title":"AutoDropout: Learning Dropout Patterns to Regularize Deep Networks","created":"2021-01-05","cited_by_count":35,"area":null},{"id":"2206.10012","title":"Limitations of the NTK for Understanding Generalization in Deep Learning","created":"2022-06-20","cited_by_count":35,"area":null},{"id":"2306.13253","title":"Predicting Grokking Long Before it Happens: A look into the loss landscape of models which grok","created":"2023-06-23","cited_by_count":35,"area":null},{"id":"2107.11228","title":"Taxonomizing local versus global structure in neural network loss landscapes","created":"2021-07-23","cited_by_count":34,"area":null},{"id":"2304.10136","title":"Diversifying the High-level Features for better Adversarial Transferability","created":"2023-04-20","cited_by_count":34,"area":null},{"id":"2205.11912","title":"Physics-Embedded Neural Networks: Graph Neural PDE Solvers with Mixed Boundary Conditions","created":"2022-05-24","cited_by_count":33,"area":null},{"id":"2201.09051","title":"On the Robustness of Sparse Counterfactual Explanations to Adverse Perturbations","created":"2022-01-22","cited_by_count":32,"area":null},{"id":"2204.02488","title":"Discovering and forecasting extreme events via active learning in neural operators","created":"2022-04-05","cited_by_count":32,"area":null},{"id":"2404.18001","title":"LLMParser: An Exploratory Study on Using Large Language Models for Log Parsing","created":"2024-04-27","cited_by_count":32,"area":null},{"id":"2110.03128","title":"On the Generalization of Models Trained with SGD: Information-Theoretic Bounds and Implications","created":"2021-10-07","cited_by_count":31,"area":null},{"id":"2202.01331","title":"Fast Convex Optimization for Two-Layer ReLU Networks: Equivalent Model Classes and Cone Decompositions","created":"2022-02-02","cited_by_count":31,"area":null},{"id":"2210.13738","title":"Pruning's Effect on Generalization Through the Lens of Training and Regularization","created":"2022-10-25","cited_by_count":31,"area":null},{"id":"2305.02657","title":"On the Eigenvalue Decay Rates of a Class of Neural-Network Related Kernel Functions Defined on General Domains","created":"2023-05-04","cited_by_count":31,"area":null},{"id":"2408.11804","title":"Approaching Deep Learning through the Spectral Dynamics of Weights","created":"2024-08-21","cited_by_count":31,"area":null},{"id":"2212.14032","title":"On Implicit Bias in Overparameterized Bilevel Optimization","created":"2022-12-28","cited_by_count":30,"area":null},{"id":"2302.05933","title":"Generalization Ability of Wide Neural Networks on $\\mathbb{R}$","created":"2023-02-12","cited_by_count":30,"area":null},{"id":"2102.07861","title":"Low Curvature Activations Reduce Overfitting in Adversarial Training","created":"2021-02-15","cited_by_count":29,"area":null},{"id":"2207.06475","title":"Analysis of Catastrophic Forgetting for Random Orthogonal Transformation Tasks in the Overparameterized Regime","created":"2022-06-01","cited_by_count":29,"area":null},{"id":"2211.05641","title":"Regression as Classification: Influence of Task Formulation on Neural Network Features","created":"2022-11-10","cited_by_count":29,"area":null},{"id":"2509.14786","title":"Pre-training under infinite compute","created":"2025-09-18","cited_by_count":29,"area":null},{"id":"2109.14509","title":"PAC-Bayes Information Bottleneck","created":"2021-09-29","cited_by_count":28,"area":null},{"id":"2209.15259","title":"On the Impossible Safety of Large AI Models","created":"2022-09-30","cited_by_count":28,"area":null},{"id":"2210.15435","title":"Grokking phase transitions in learning local rules with gradient descent","created":"2022-10-26","cited_by_count":28,"area":null},{"id":"2409.15771","title":"Zero-shot forecasting of chaotic systems","created":"2024-09-24","cited_by_count":28,"area":null},{"id":"2102.13229","title":"Consistent Sparse Deep Learning: Theory and Computation","created":"2021-02-25","cited_by_count":27,"area":null},{"id":"2201.12150","title":"Learning Curves for Decision Making in Supervised Machine Learning: A Survey","created":"2022-01-28","cited_by_count":27,"area":null},{"id":"2202.05189","title":"Understanding Rare Spurious Correlations in Neural Networks","created":"2022-02-10","cited_by_count":27,"area":null},{"id":"2309.16314","title":"A Primer on Bayesian Neural Networks: Review and Debates","created":"2023-09-28","cited_by_count":27,"area":null},{"id":"2205.12808","title":"Mirror Descent Maximizes Generalized Margin and Can Be Implemented Efficiently","created":"2022-05-25","cited_by_count":26,"area":null},{"id":"2206.08684","title":"Sparse Double Descent: Where Network Pruning Aggravates Overfitting","created":"2022-06-17","cited_by_count":26,"area":null},{"id":"2302.10886","title":"Some Fundamental Aspects about Lipschitz Continuity of Neural Networks","created":"2023-02-21","cited_by_count":26,"area":null},{"id":"2304.09595","title":"AdapterGNN: Parameter-Efficient Fine-Tuning Improves Generalization in GNNs","created":"2023-04-19","cited_by_count":26,"area":null},{"id":"2312.03656","title":"Interpretability Illusions in the Generalization of Simplified Models","created":"2023-12-06","cited_by_count":25,"area":null}],"next_cursor":"100"}