{"list":"other","total":55,"rows":[{"id":"1512.03385","title":"Deep Residual Learning for Image Recognition","created":"2015-12-10","cited_by_count":52096},{"id":"1810.04805","title":"BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding","created":"2018-10-11","cited_by_count":40399},{"id":"2103.00020","title":"Learning Transferable Visual Models From Natural Language Supervision","created":"2021-02-26","cited_by_count":33744},{"id":"2005.14165","title":"Language Models are Few-Shot Learners","created":"2020-05-28","cited_by_count":32419},{"id":"2001.08361","title":"Scaling Laws for Neural Language Models","created":"2020-01-23","cited_by_count":6586},{"id":"2006.11477","title":"wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations","created":"2020-06-20","cited_by_count":3905},{"id":"2206.07682","title":"Emergent Abilities of Large Language Models","created":"2022-06-15","cited_by_count":2512},{"id":"2206.04615","title":"Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models","created":"2022-06-09","cited_by_count":1998},{"id":"1812.11118","title":"Reconciling modern machine learning practice and the bias-variance trade-off","created":"2018-12-28","cited_by_count":927},{"id":"1503.02406","title":"Deep Learning and the Information Bottleneck Principle","created":"2015-03-09","cited_by_count":802},{"id":"1912.11370","title":"Big Transfer (BiT): General Visual Representation Learning","created":"2019-12-24","cited_by_count":769},{"id":"1912.02292","title":"Deep Double Descent: Where Bigger Models and More Data Hurt","created":"2019-12-04","cited_by_count":672},{"id":"1810.02054","title":"Gradient Descent Provably Optimizes Over-parameterized Neural Networks","created":"2018-10-04","cited_by_count":659},{"id":"2201.02177","title":"Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets","created":"2022-01-06","cited_by_count":629},{"id":"1802.10026","title":"Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNs","created":"2018-02-27","cited_by_count":588},{"id":"2302.06675","title":"Symbolic Discovery of Optimization Algorithms","created":"2023-02-13","cited_by_count":434},{"id":"2103.00065","title":"Gradient Descent on Neural Networks Typically Occurs at the Edge of Stability","created":"2021-02-26","cited_by_count":427},{"id":"1611.01838","title":"Entropy-SGD: Biasing Gradient Descent Into Wide Valleys","created":"2016-11-06","cited_by_count":359},{"id":"1803.00885","title":"Essentially No Barriers in Neural Network Energy Landscape","created":"2018-03-02","cited_by_count":330},{"id":"1704.04289","title":"Stochastic Gradient Descent as Approximate Bayesian Inference","created":"2017-04-13","cited_by_count":300},{"id":"1711.04623","title":"Three Factors Influencing Minima in SGD","created":"2017-11-13","cited_by_count":273},{"id":"2205.10343","title":"Towards Understanding Grokking: An Effective Theory of Representation Learning","created":"2022-05-20","cited_by_count":230},{"id":"2101.12176","title":"On the Origin of Implicit Regularization in Stochastic Gradient Descent","created":"2021-01-28","cited_by_count":184},{"id":"1907.04595","title":"Towards Explaining the Regularization Effect of Initial Large Learning Rate in Training Neural Networks","created":"2019-07-10","cited_by_count":162},{"id":"1712.06559","title":"The Power of Interpolation: Understanding the Effectiveness of SGD in Modern Over-parametrized Learning","created":"2017-12-18","cited_by_count":150},{"id":"2009.11162","title":"Implicit Gradient Regularization","created":"2020-09-23","cited_by_count":146},{"id":"1811.08888","title":"Stochastic Gradient Descent Optimizes Over-parameterized Deep ReLU Networks","created":"2018-11-21","cited_by_count":141},{"id":"1710.06451","title":"A Bayesian Perspective on Generalization and Stochastic Gradient Descent","created":"2017-10-17","cited_by_count":135},{"id":"2309.07311","title":"Sudden Drops in the Loss: Syntax Acquisition, Phase Transitions, and Simplicity Bias in MLMs","created":"2023-09-13","cited_by_count":124},{"id":"1710.11029","title":"Stochastic gradient descent performs variational inference, converges to limit cycles for deep networks","created":"2017-10-30","cited_by_count":109},{"id":"2006.15081","title":"On the Generalization Benefit of Noise in Stochastic Gradient Descent","created":"2020-06-26","cited_by_count":82},{"id":"2310.04415","title":"Why Do We Need Weight Decay in Modern Deep Learning?","created":"2023-10-06","cited_by_count":72},{"id":"2210.05337","title":"SGD with Large Step Sizes Learns Sparse Features","created":"2022-10-11","cited_by_count":68},{"id":"1806.01796","title":"Stochastic Gradient Descent on Separable Data: Exact Convergence with a Fixed Learning Rate","created":"2018-06-05","cited_by_count":64},{"id":"2109.14119","title":"Stochastic Training is Not Necessary for Generalization","created":"2021-09-29","cited_by_count":63},{"id":"1810.00004","title":"Fluctuation-dissipation relations for stochastic gradient descent","created":"2018-09-28","cited_by_count":48},{"id":"2001.07384","title":"Understanding Why Neural Networks Generalize Well Through GSNR of Parameters","created":"2020-01-21","cited_by_count":40},{"id":"2102.05375","title":"Strength of Minibatch Noise in SGD","created":"2021-02-10","cited_by_count":39},{"id":"2306.04251","title":"Stochastic Collapse: How Gradient Noise Attracts SGD Dynamics Towards Simpler Subnetworks","created":"2023-06-07","cited_by_count":39},{"id":"2012.03636","title":"Noise and Fluctuation of Finite Learning Rate Stochastic Gradient Descent","created":"2020-12-07","cited_by_count":37},{"id":"2402.15017","title":"Towards Few-Shot Adaptation of Foundation Models via Multitask Finetuning","created":"2024-02-22","cited_by_count":31},{"id":"2401.01736","title":"Few-shot Adaptation of Multi-modal Foundation Models: A Survey","created":"2024-01-03","cited_by_count":28},{"id":"1803.01927","title":"Energy-entropy competition and the effectiveness of stochastic gradient descent in machine learning","created":"2018-03-05","cited_by_count":20},{"id":"2106.15739","title":"On the Periodic Behavior of Neural Network Training with Batch Normalization and Weight Decay","created":"2021-06-29","cited_by_count":20},{"id":"2107.09133","title":"The Limiting Dynamics of SGD: Modified Loss, Phase Space Oscillations, and Anomalous Diffusion","created":"2021-07-19","cited_by_count":19},{"id":"2209.03695","title":"Training Scale-Invariant Neural Networks on the Sphere Can Happen in Three Regimes","created":"2022-09-08","cited_by_count":19},{"id":"2309.10688","title":"On the different regimes of Stochastic Gradient Descent","created":"2023-09-19","cited_by_count":12},{"id":"2410.22244","title":"Abrupt Learning in Transformers: A Case Study on Matrix Completion","created":"2024-10-29","cited_by_count":11},{"id":"1808.10474","title":"Fixed-Time Stable Gradient Flows: Applications to Continuous-Time Optimization","created":"2018-08-30","cited_by_count":10},{"id":"1807.04162","title":"TherML: Thermodynamics of Machine Learning","created":"2018-07-11","cited_by_count":9},{"id":"2305.06435","title":"Phase transitions in the mini-batch size for sparse and dense two-layer neural networks","created":"2023-05-10","cited_by_count":9},{"id":"2002.12406","title":"A Free-Energy Principle for Representation Learning","created":"2020-02-27","cited_by_count":7},{"id":"2410.22113","title":"Where Do Large Learning Rates Lead Us?","created":"2024-10-29","cited_by_count":6},{"id":"2303.15739","title":"Bayesian Free Energy of Deep ReLU Neural Network in Overparametrized Cases","created":"2023-03-28","cited_by_count":5},{"id":"2309.13681","title":"Accelerating Large Batch Training via Gradient Signal to Noise Ratio (GSNR)","created":"2023-09-24","cited_by_count":3}],"next_cursor":null}