{"list":"other","total":58,"rows":[{"id":"1706.03762","title":"Attention Is All You Need","created":"2017-06-12","cited_by_count":60361},{"id":"2005.14165","title":"Language Models are Few-Shot Learners","created":"2020-05-28","cited_by_count":32419},{"id":"2302.13971","title":"LLaMA: Open and Efficient Foundation Language Models","created":"2023-02-27","cited_by_count":14132},{"id":"2104.09864","title":"RoFormer: Enhanced Transformer with Rotary Position Embedding","created":"2021-04-20","cited_by_count":4703},{"id":"1301.3781","title":"Efficient Estimation of Word Representations in Vector Space","created":"2013-01-16","cited_by_count":4375},{"id":"2202.12837","title":"Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?","created":"2022-02-25","cited_by_count":1434},{"id":"2209.11895","title":"In-context Learning and Induction Heads","created":"2022-09-24","cited_by_count":964},{"id":"2111.02080","title":"An Explanation of In-context Learning as Implicit Bayesian Inference","created":"2021-11-03","cited_by_count":938},{"id":"2212.07677","title":"Transformers learn in-context by gradient descent","created":"2022-12-15","cited_by_count":732},{"id":"2208.01066","title":"What Can Transformers Learn In-Context? A Case Study of Simple Function Classes","created":"2022-08-01","cited_by_count":715},{"id":"2211.15661","title":"What learning algorithm is in-context learning? Investigations with linear models","created":"2022-11-28","cited_by_count":684},{"id":"2306.09927","title":"Trained Transformers Learn Linear Models In-Context","created":"2023-06-16","cited_by_count":337},{"id":"2205.05055","title":"Data Distributional Properties Drive Emergent In-Context Learning in Transformers","created":"2022-04-22","cited_by_count":309},{"id":"2310.15213","title":"Function Vectors in Large Language Models","created":"2023-10-23","cited_by_count":308},{"id":"2306.04637","title":"Transformers as Statisticians: Provable In-Context Learning with In-Context Algorithm Selection","created":"2023-06-07","cited_by_count":298},{"id":"2306.00297","title":"Transformers learn to implement preconditioned gradient descent for in-context learning","created":"2023-06-01","cited_by_count":279},{"id":"2301.07067","title":"Transformers as Algorithms: Generalization and Stability in In-context Learning","created":"2023-01-17","cited_by_count":222},{"id":"2307.03576","title":"One Step of Gradient Descent is Provably the Optimal In-Context Learner with One Layer of Linear Self-Attention","created":"2023-07-07","cited_by_count":170},{"id":"2306.00802","title":"Birth of a Transformer: A Memory Viewpoint","created":"2023-06-01","cited_by_count":161},{"id":"2203.16634","title":"Transformer Language Models without Positional Encodings Still Learn Positional Information","created":"2022-03-30","cited_by_count":150},{"id":"2301.11916","title":"Large Language Models Are Latent Variable Models: Explaining and Finding Good Demonstrations for In-Context Learning","created":"2023-01-27","cited_by_count":149},{"id":"2306.15063","title":"Pretraining task diversity and the emergence of non-Bayesian in-context learning for regression","created":"2023-06-26","cited_by_count":146},{"id":"2303.07895","title":"The Learnability of In-Context Learning","created":"2023-03-14","cited_by_count":127},{"id":"2305.19420","title":"What and How does In-Context Learning Learn? Bayesian Model Averaging, Parameterization, and Generalization","created":"2023-05-30","cited_by_count":118},{"id":"2310.05249","title":"In-Context Convergence of Transformers","created":"2023-10-08","cited_by_count":115},{"id":"2302.13539","title":"Finding Support Examples for In-Context Learning","created":"2023-02-27","cited_by_count":104},{"id":"2310.08391","title":"How Many Pretraining Tasks Are Needed for In-Context Learning of Linear Regression?","created":"2023-10-12","cited_by_count":96},{"id":"2401.12973","title":"In-Context Language Learning: Architectures and Algorithms","created":"2024-01-23","cited_by_count":89},{"id":"2303.07971","title":"A Theory of Emergent In-Context Learning as Implicit Structure Induction","created":"2023-03-14","cited_by_count":86},{"id":"2306.04891","title":"In-Context Learning through the Bayesian Prism","created":"2023-06-08","cited_by_count":83},{"id":"2311.08360","title":"The Transient Nature of Emergent In-Context Learning in Transformers","created":"2023-11-14","cited_by_count":76},{"id":"2310.10616","title":"How Do Transformers Learn In-Context Beyond Simple Functions? A Case Study on Learning with Representations","created":"2023-10-16","cited_by_count":75},{"id":"2310.09881","title":"In-Context Learning with Iterative Demonstration Selection","created":"2023-10-15","cited_by_count":74},{"id":"2402.19442","title":"Training Dynamics of Multi-Head Softmax Attention for In-Context Learning: Emergence, Convergence, and Optimality","created":"2024-02-29","cited_by_count":74},{"id":"2310.03016","title":"Understanding In-Context Learning in Transformers and LLMs by Learning to Learn Discrete Functions","created":"2023-10-04","cited_by_count":63},{"id":"2401.12087","title":"Revisiting Demonstration Selection Strategies in In-Context Learning","created":"2024-01-22","cited_by_count":58},{"id":"2307.12375","title":"In-Context Learning Learns Label Relationships but Is Not Conventional Learning","created":"2023-07-23","cited_by_count":51},{"id":"2402.13055","title":"Identifying Semantic Induction Heads to Understand In-Context Learning","created":"2024-02-20","cited_by_count":47},{"id":"2311.00871","title":"Pretraining Data Mixtures Enable Narrow Model Selection Capabilities in Transformer Models","created":"2023-11-01","cited_by_count":45},{"id":"2402.18819","title":"Dual Operating Modes of In-Context Learning","created":"2024-02-29","cited_by_count":43},{"id":"2304.13276","title":"The Closeness of In-Context Learning and Weight Shifting for Softmax Regression","created":"2023-04-26","cited_by_count":41},{"id":"2306.15091","title":"Understanding In-Context Learning via Supportive Pretraining Data","created":"2023-06-26","cited_by_count":41},{"id":"2401.15530","title":"An Information-Theoretic Analysis of In-Context Learning","created":"2024-01-28","cited_by_count":40},{"id":"2310.00297","title":"Understanding In-Context Learning from Repetitions","created":"2023-09-30","cited_by_count":29},{"id":"2402.14180","title":"Linear Transformers are Versatile In-Context Learners","created":"2024-02-21","cited_by_count":28},{"id":"2308.06912","title":"CausalLM is not optimal for in-context learning","created":"2023-08-14","cited_by_count":26},{"id":"2402.12530","title":"Parallel Structures in Pre-training Data Yield In-Context Learning","created":"2024-02-19","cited_by_count":26},{"id":"2305.16704","title":"A Closer Look at In-Context Learning under Distribution Shifts","created":"2023-05-26","cited_by_count":25},{"id":"2305.12766","title":"Understanding Emergent In-Context Learning from a Kernel Regression Perspective","created":"2023-05-22","cited_by_count":24},{"id":"2402.11639","title":"In-Context Learning with Transformers: Softmax Attention Adapts to Function Lipschitzness","created":"2024-02-18","cited_by_count":22},{"id":"2402.02872","title":"How do Large Language Models Learn In-Context? Query and Key Matrices of In-Context Heads are Two Towers for Metric Learning","created":"2024-02-05","cited_by_count":18},{"id":"2305.17040","title":"A Mechanism for Sample-Efficient In-Context Learning for Sparse Retrieval Tasks","created":"2023-05-26","cited_by_count":17},{"id":"2401.17426","title":"Superiority of Multi-Head Attention in In-Context Linear Regression","created":"2024-01-30","cited_by_count":17},{"id":"2402.05787","title":"How do Transformers perform In-Context Autoregressive Learning?","created":"2024-02-08","cited_by_count":17},{"id":"2401.12406","title":"Enhancing In-context Learning via Linear Probe Calibration","created":"2024-01-22","cited_by_count":16},{"id":"2307.01201","title":"Schema-learning and rebinding as mechanisms of in-context learning and emergence","created":"2023-06-16","cited_by_count":14},{"id":"2402.05515","title":"NoisyICL: A Little Noise in Model Parameters Calibrates In-context Learning","created":"2024-02-08","cited_by_count":4},{"id":"2402.10424","title":"Pelican Soup Framework: A Theoretical Framework for Language Model Capabilities","created":"2024-02-16","cited_by_count":2}],"next_cursor":null}