{"list":"other","total":58,"rows":[{"id":"2005.14165","title":"Language Models are Few-Shot Learners","created":"2020-05-28","cited_by_count":32419},{"id":"2307.03172","title":"Lost in the Middle: How Language Models Use Long Contexts","created":"2023-07-06","cited_by_count":3455},{"id":"2206.07682","title":"Emergent Abilities of Large Language Models","created":"2022-06-15","cited_by_count":2512},{"id":"2206.04615","title":"Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models","created":"2022-06-09","cited_by_count":1998},{"id":"2202.12837","title":"Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?","created":"2022-02-25","cited_by_count":1434},{"id":"2209.11895","title":"In-context Learning and Induction Heads","created":"2022-09-24","cited_by_count":964},{"id":"2111.02080","title":"An Explanation of In-context Learning as Implicit Bayesian Inference","created":"2021-11-03","cited_by_count":938},{"id":"2301.05217","title":"Progress measures for grokking via mechanistic interpretability","created":"2023-01-12","cited_by_count":839},{"id":"2212.07677","title":"Transformers learn in-context by gradient descent","created":"2022-12-15","cited_by_count":732},{"id":"2208.01066","title":"What Can Transformers Learn In-Context? A Case Study of Simple Function Classes","created":"2022-08-01","cited_by_count":715},{"id":"1710.10345","title":"The Implicit Bias of Gradient Descent on Separable Data","created":"2017-10-27","cited_by_count":655},{"id":"2201.02177","title":"Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets","created":"2022-01-06","cited_by_count":629},{"id":"2402.00838","title":"OLMo: Accelerating the Science of Language Models","created":"2024-02-01","cited_by_count":613},{"id":"2304.15004","title":"Are Emergent Abilities of Large Language Models a Mirage?","created":"2023-04-28","cited_by_count":523},{"id":"2006.07710","title":"The Pitfalls of Simplicity Bias in Neural Networks","created":"2020-06-13","cited_by_count":351},{"id":"1704.04289","title":"Stochastic Gradient Descent as Approximate Bayesian Inference","created":"2017-04-13","cited_by_count":300},{"id":"2011.09468","title":"Gradient Starvation: A Learning Proclivity in Neural Networks","created":"2020-11-18","cited_by_count":267},{"id":"2301.07067","title":"Transformers as Algorithms: Generalization and Stability in In-context Learning","created":"2023-01-17","cited_by_count":222},{"id":"2202.07785","title":"Predictability and Surprise in Large Generative Models","created":"2022-02-15","cited_by_count":210},{"id":"2312.04927","title":"Zoology: Measuring and Improving Recall in Efficient Language Models","created":"2023-12-08","cited_by_count":204},{"id":"2306.17844","title":"The Clock and the Pizza: Two Stories in Mechanistic Explanation of Neural Networks","created":"2023-06-30","cited_by_count":156},{"id":"2210.01117","title":"Omnigrok: Grokking Beyond Algorithmic Data","created":"2022-10-03","cited_by_count":146},{"id":"2306.15063","title":"Pretraining task diversity and the emergence of non-Bayesian in-context learning for regression","created":"2023-06-26","cited_by_count":146},{"id":"2303.13506","title":"The Quantization Model of Neural Scaling","created":"2023-03-23","cited_by_count":138},{"id":"2309.07311","title":"Sudden Drops in the Loss: Syntax Acquisition, Phase Transitions, and Simplicity Bias in MLMs","created":"2023-09-13","cited_by_count":124},{"id":"2312.03002","title":"The mechanistic basis of data dependence and abrupt learning in an in-context classification task","created":"2023-12-03","cited_by_count":118},{"id":"2402.11004","title":"The Evolution of Statistical Induction Heads: In-Context Learning Markov Chains","created":"2024-02-16","cited_by_count":118},{"id":"2309.01809","title":"Are Emergent Abilities in Large Language Models just In-Context Learning?","created":"2023-09-04","cited_by_count":109},{"id":"2310.06110","title":"Grokking as the Transition from Lazy to Rich Training Dynamics","created":"2023-10-09","cited_by_count":108},{"id":"2310.08391","title":"How Many Pretraining Tasks Are Needed for In-Context Learning of Linear Regression?","created":"2023-10-12","cited_by_count":96},{"id":"2403.15796","title":"Understanding Emergent Abilities of Language Models from the Loss Perspective","created":"2024-03-23","cited_by_count":94},{"id":"2401.12973","title":"In-Context Language Learning: Architectures and Algorithms","created":"2024-01-23","cited_by_count":89},{"id":"2404.07129","title":"What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation","created":"2024-04-10","cited_by_count":89},{"id":"2301.02679","title":"Grokking modular arithmetic","created":"2023-01-06","cited_by_count":83},{"id":"2311.18817","title":"Dichotomy of Early and Late Phase Implicit Biases Can Provably Induce Grokking","created":"2023-11-30","cited_by_count":80},{"id":"2305.08298","title":"Symbol tuning improves in-context learning in language models","created":"2023-05-15","cited_by_count":75},{"id":"2502.14010","title":"Which Attention Heads Matter for In-Context Learning?","created":"2025-02-19","cited_by_count":61},{"id":"2406.16778","title":"Finding Transformer Circuits with Edge Pruning","created":"2024-06-24","cited_by_count":60},{"id":"2308.12108","title":"The Local Learning Coefficient: A Singularity-Aware Complexity Measure","created":"2023-08-23","cited_by_count":51},{"id":"2402.15555","title":"Deep Networks Always Grok and Here is Why","created":"2024-02-23","cited_by_count":51},{"id":"2006.15191","title":"Is SGD a Bayesian sampler? Well, almost","created":"2020-06-26","cited_by_count":50},{"id":"2303.06173","title":"Unifying Grokking and Double Descent","created":"2023-03-10","cited_by_count":50},{"id":"2501.04697","title":"Grokking at the Edge of Numerical Stability","created":"2025-01-08","cited_by_count":46},{"id":"2501.16265","title":"Training Dynamics of In-Context Learning in Linear Attention","created":"2025-01-27","cited_by_count":46},{"id":"2402.18819","title":"Dual Operating Modes of In-Context Learning","created":"2024-02-29","cited_by_count":43},{"id":"2310.06301","title":"Dynamical versus Bayesian Phase Transitions in a Toy Model of Superposition","created":"2023-10-10","cited_by_count":29},{"id":"2407.20199","title":"Emergence in non-neural models: grokking modular arithmetic via average gradient outer product","created":"2024-07-29","cited_by_count":28},{"id":"2407.17686","title":"Transformers on Markov Data: Constant Depth Suffices","created":"2024-07-25","cited_by_count":27},{"id":"2408.12578","title":"A Percolation Model of Emergence: Analyzing Transformers Trained on a Formal Language","created":"2024-08-22","cited_by_count":26},{"id":"2112.03215","title":"Multi-scale Feature Learning Dynamics: Insights for Double Descent","created":"2021-12-06","cited_by_count":24},{"id":"2405.14578","title":"Surge Phenomenon in Optimal Learning Rate and Batch Size Scaling","created":"2024-05-23","cited_by_count":22},{"id":"2412.00104","title":"Differential learning kinetics govern the transition from memorization to generalization during in-context learning","created":"2024-11-27","cited_by_count":21},{"id":"2402.05787","title":"How do Transformers perform In-Context Autoregressive Learning?","created":"2024-02-08","cited_by_count":17},{"id":"2308.08536","title":"Can Transformers Learn Optimal Filtering for Unknown Systems?","created":"2023-08-16","cited_by_count":16},{"id":"2406.00053","title":"Dual Process Learning: Controlling Use of In-Context vs. In-Weights Strategies with Weight Forgetting","created":"2024-05-28","cited_by_count":15},{"id":"2406.14022","title":"Investigating the Pre-Training Dynamics of In-Context Learning: Task Recognition vs. Task Learning","created":"2024-06-20","cited_by_count":6},{"id":"1709.06493","title":"Learning to update Auto-associative Memory in Recurrent Neural Networks for Improving Sequence Memorization","created":"2017-09-19","cited_by_count":5},{"id":"2504.12916","title":"Exact Learning Dynamics of In-Context Learning in Linear Transformers and Its Application to Non-Linear Transformers","created":"2025-04-17","cited_by_count":3}],"next_cursor":null}