{"list":"other","total":73,"rows":[{"id":"1706.03762","title":"Attention Is All You Need","created":"2017-06-12","cited_by_count":60361},{"id":"2005.14165","title":"Language Models are Few-Shot Learners","created":"2020-05-28","cited_by_count":32419},{"id":"2312.00752","title":"Mamba: Linear-Time Sequence Modeling with Selective State Spaces","created":"2023-12-01","cited_by_count":4718},{"id":"2206.07682","title":"Emergent Abilities of Large Language Models","created":"2022-06-15","cited_by_count":2512},{"id":"2206.04615","title":"Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models","created":"2022-06-09","cited_by_count":1998},{"id":"2202.12837","title":"Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?","created":"2022-02-25","cited_by_count":1434},{"id":"2101.06804","title":"What Makes Good In-Context Examples for GPT-$3$?","created":"2021-01-17","cited_by_count":1195},{"id":"2209.11895","title":"In-context Learning and Induction Heads","created":"2022-09-24","cited_by_count":964},{"id":"2111.02080","title":"An Explanation of In-context Learning as Implicit Bayesian Inference","created":"2021-11-03","cited_by_count":938},{"id":"1706.05098","title":"An Overview of Multi-Task Learning in Deep Neural Networks","created":"2017-06-15","cited_by_count":803},{"id":"2003.11080","title":"XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalization","created":"2020-03-24","cited_by_count":742},{"id":"2212.07677","title":"Transformers learn in-context by gradient descent","created":"2022-12-15","cited_by_count":732},{"id":"2208.01066","title":"What Can Transformers Learn In-Context? A Case Study of Simple Function Classes","created":"2022-08-01","cited_by_count":715},{"id":"2211.15661","title":"What learning algorithm is in-context learning? Investigations with linear models","created":"2022-11-28","cited_by_count":684},{"id":"2112.08633","title":"Learning To Retrieve Prompts for In-Context Learning","created":"2021-12-16","cited_by_count":648},{"id":"2306.11644","title":"Textbooks Are All You Need","created":"2023-06-20","cited_by_count":569},{"id":"2102.07078","title":"Exploiting Shared Representations for Personalized Federated Learning","created":"2021-02-14","cited_by_count":541},{"id":"1907.05388","title":"Provably Efficient Reinforcement Learning with Linear Function Approximation","created":"2019-07-11","cited_by_count":520},{"id":"2110.15943","title":"MetaICL: Learning to Learn In Context","created":"2021-10-29","cited_by_count":454},{"id":"2303.03846","title":"Larger language models do in-context learning differently","created":"2023-03-07","cited_by_count":394},{"id":"2302.10866","title":"Hyena Hierarchy: Towards Larger Convolutional Language Models","created":"2023-02-21","cited_by_count":393},{"id":"2009.09796","title":"Multi-Task Learning with Deep Neural Networks: A Survey","created":"2020-09-10","cited_by_count":375},{"id":"2306.09927","title":"Trained Transformers Learn Linear Models In-Context","created":"2023-06-16","cited_by_count":337},{"id":"2205.05055","title":"Data Distributional Properties Drive Emergent In-Context Learning in Transformers","created":"2022-04-22","cited_by_count":309},{"id":"2310.15213","title":"Function Vectors in Large Language Models","created":"2023-10-23","cited_by_count":308},{"id":"2306.04637","title":"Transformers as Statisticians: Provable In-Context Learning with In-Context Algorithm Selection","created":"2023-06-07","cited_by_count":298},{"id":"2306.00297","title":"Transformers learn to implement preconditioned gradient descent for in-context learning","created":"2023-06-01","cited_by_count":279},{"id":"2002.09434","title":"Few-Shot Learning via Learning the Representation, Provably","created":"2020-02-21","cited_by_count":214},{"id":"1905.11604","title":"SGD on Neural Networks Learns Functions of Increasing Complexity","created":"2019-05-28","cited_by_count":204},{"id":"1505.06279","title":"The Benefit of Multitask Representation Learning","created":"2015-05-23","cited_by_count":195},{"id":"2006.11650","title":"On the Theory of Transfer Learning: The Importance of Task Diversity","created":"2020-06-20","cited_by_count":186},{"id":"2307.03576","title":"One Step of Gradient Descent is Provably the Optimal In-Context Learner with One Layer of Linear Self-Attention","created":"2023-07-07","cited_by_count":170},{"id":"2002.11684","title":"Provable Meta-Learning of Linear Representations","created":"2020-02-26","cited_by_count":161},{"id":"2206.15144","title":"Neural Networks can Learn Representations with Gradient Descent","created":"2022-06-30","cited_by_count":161},{"id":"2306.15063","title":"Pretraining task diversity and the emergence of non-Bayesian in-context learning for regression","created":"2023-06-26","cited_by_count":146},{"id":"2312.03002","title":"The mechanistic basis of data dependence and abrupt learning in an in-context classification task","created":"2023-12-03","cited_by_count":118},{"id":"2206.04301","title":"Unveiling Transformers with LEGO: a synthetic reasoning task","created":"2022-06-09","cited_by_count":106},{"id":"2212.04458","title":"General-Purpose In-Context Learning by Meta-Learning Transformers","created":"2022-12-08","cited_by_count":105},{"id":"2402.04248","title":"Can Mamba Learn How to Learn? A Comparative Study on In-Context Learning Tasks","created":"2024-02-06","cited_by_count":105},{"id":"2309.05858","title":"Uncovering mesa-optimization algorithms in Transformers","created":"2023-09-11","cited_by_count":100},{"id":"2210.15651","title":"Learning Single-Index Models with Shallow Neural Networks","created":"2022-10-27","cited_by_count":92},{"id":"2401.12973","title":"In-Context Language Learning: Architectures and Algorithms","created":"2024-01-23","cited_by_count":89},{"id":"2404.07129","title":"What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation","created":"2024-04-10","cited_by_count":89},{"id":"2310.08566","title":"Transformers as Decision Makers: Provable In-Context Reinforcement Learning via Supervised Pretraining","created":"2023-10-12","cited_by_count":83},{"id":"2310.10616","title":"How Do Transformers Learn In-Context Beyond Simple Functions? A Case Study on Learning with Representations","created":"2023-10-16","cited_by_count":75},{"id":"2402.19442","title":"Training Dynamics of Multi-Head Softmax Attention for In-Context Learning: Emergence, Convergence, and Optimality","created":"2024-02-29","cited_by_count":74},{"id":"2305.18270","title":"How Two-Layer Neural Networks Learn, One (Giant) Step at a Time","created":"2023-05-29","cited_by_count":68},{"id":"2310.03016","title":"Understanding In-Context Learning in Transformers and LLMs by Learning to Learn Discrete Functions","created":"2023-10-04","cited_by_count":63},{"id":"2312.06528","title":"Transformers Implement Functional Gradient Descent to Learn Non-Linear Functions In Context","created":"2023-12-11","cited_by_count":63},{"id":"2303.00055","title":"Learning time-scales in two-layers neural networks","created":"2023-02-28","cited_by_count":57},{"id":"2010.06531","title":"Impact of Representation Learning in Linear Bandits","created":"2020-10-13","cited_by_count":52},{"id":"2211.11567","title":"Neural networks trained with SGD learn distributions of increasing complexity","created":"2022-11-21","cited_by_count":50},{"id":"2102.04132","title":"Near-optimal Representation Learning for Linear Bandits and Linear RL","created":"2021-02-08","cited_by_count":45},{"id":"2402.03170","title":"Is Mamba Capable of In-Context Learning?","created":"2024-02-05","cited_by_count":45},{"id":"2405.18634","title":"A Theoretical Understanding of Self-Correction through In-context Alignment","created":"2024-05-28","cited_by_count":44},{"id":"2402.18819","title":"Dual Operating Modes of In-Context Learning","created":"2024-02-29","cited_by_count":43},{"id":"2204.03508","title":"A Survey of Multi-task Learning in Natural Language Processing: Regarding Task Relatedness and Training Methods","created":"2022-04-07","cited_by_count":42},{"id":"2301.11500","title":"Understanding Incremental Learning of Gradient Descent: A Fine-grained Analysis of Matrix Sensing","created":"2023-01-27","cited_by_count":42},{"id":"2406.02550","title":"Learning to grok: Emergence of in-context learning and skill composition in modular arithmetic tasks","created":"2024-06-04","cited_by_count":40},{"id":"2106.08053","title":"On the Power of Multitask Representation Learning in Linear MDP","created":"2021-06-15","cited_by_count":30},{"id":"2211.06506","title":"Spectral Evolution and Invariance in Linear-width Neural Networks","created":"2022-11-11","cited_by_count":30},{"id":"2206.05900","title":"Provable Benefit of Multitask Representation Learning in Reinforcement Learning","created":"2022-06-13","cited_by_count":29},{"id":"2408.09503","title":"Out-of-distribution generalization via composition: a lens through induction heads in Transformers","created":"2024-08-18","cited_by_count":28},{"id":"2403.03183","title":"How Well Can Transformers Emulate In-context Newton's Method?","created":"2024-03-05","cited_by_count":25},{"id":"2212.02216","title":"Improving Few-Shot Performance of Language Models via Nearest Neighbor Calibration","created":"2022-12-05","cited_by_count":24},{"id":"2111.07167","title":"The Three Stages of Learning Dynamics in High-Dimensional Kernel Methods","created":"2021-11-13","cited_by_count":22},{"id":"1701.02962","title":"Distinguishing Antonyms and Synonyms in a Pattern-based Neural Network","created":"2017-01-11","cited_by_count":19},{"id":"2305.12467","title":"Understanding Multi-phase Optimization Dynamics and Rich Nonlinear Behaviors of ReLU Networks","created":"2023-05-21","cited_by_count":15},{"id":"1908.11813","title":"Multi-Task Learning with Language Modeling for Question Generation","created":"2019-08-30","cited_by_count":10},{"id":"2309.06054","title":"Breaking through the learning plateaus of in-context learning in Transformer","created":"2023-09-12","cited_by_count":8},{"id":"2209.03447","title":"Blessing of Class Diversity in Pre-training","created":"2022-09-07","cited_by_count":4},{"id":"2306.04551","title":"Multi-Task Training with In-Domain Language Models for Diagnostic Reasoning","created":"2023-06-07","cited_by_count":4},{"id":"2406.17467","title":"Early learning of the optimal constant solution in neural networks and humans","created":"2024-06-25","cited_by_count":3}],"next_cursor":null}