{"list":"other","total":52,"rows":[{"id":"1706.03762","title":"Attention Is All You Need","created":"2017-06-12","cited_by_count":60361},{"id":"2005.14165","title":"Language Models are Few-Shot Learners","created":"2020-05-28","cited_by_count":32419},{"id":"2303.08774","title":"GPT-4 Technical Report","created":"2023-03-15","cited_by_count":18396},{"id":"2203.02155","title":"Training language models to follow instructions with human feedback","created":"2022-03-04","cited_by_count":15971},{"id":"2302.13971","title":"LLaMA: Open and Efficient Foundation Language Models","created":"2023-02-27","cited_by_count":14132},{"id":"2307.09288","title":"Llama 2: Open Foundation and Fine-Tuned Chat Models","created":"2023-07-18","cited_by_count":13028},{"id":"1910.03771","title":"HuggingFace's Transformers: State-of-the-art Natural Language Processing","created":"2019-10-09","cited_by_count":2218},{"id":"1610.01644","title":"Understanding intermediate layers using linear classifier probes","created":"2016-10-05","cited_by_count":1353},{"id":"1608.08710","title":"Pruning Filters for Efficient ConvNets","created":"2016-08-31","cited_by_count":1143},{"id":"2111.02080","title":"An Explanation of In-context Learning as Implicit Bayesian Inference","created":"2021-11-03","cited_by_count":938},{"id":"2212.07677","title":"Transformers learn in-context by gradient descent","created":"2022-12-15","cited_by_count":732},{"id":"2208.01066","title":"What Can Transformers Learn In-Context? A Case Study of Simple Function Classes","created":"2022-08-01","cited_by_count":715},{"id":"2211.15661","title":"What learning algorithm is in-context learning? Investigations with linear models","created":"2022-11-28","cited_by_count":684},{"id":"1906.11300","title":"Benign Overfitting in Linear Regression","created":"2019-06-26","cited_by_count":549},{"id":"2403.03853","title":"ShortGPT: Layers in Large Language Models are More Redundant Than You Expect","created":"2024-03-06","cited_by_count":383},{"id":"2306.09927","title":"Trained Transformers Learn Linear Models In-Context","created":"2023-06-16","cited_by_count":337},{"id":"2306.04637","title":"Transformers as Statisticians: Provable In-Context Learning with In-Context Algorithm Selection","created":"2023-06-07","cited_by_count":298},{"id":"2306.00297","title":"Transformers learn to implement preconditioned gradient descent for in-context learning","created":"2023-06-01","cited_by_count":279},{"id":"2309.14316","title":"Physics of Language Models: Part 3.1, Knowledge Storage and Extraction","created":"2023-09-25","cited_by_count":270},{"id":"2210.10749","title":"Transformers Learn Shortcuts to Automata","created":"2022-10-19","cited_by_count":268},{"id":"2301.13196","title":"Looped Transformers as Programmable Computers","created":"2023-01-30","cited_by_count":237},{"id":"2403.17887","title":"The Unreasonable Ineffectiveness of the Deeper Layers","created":"2024-03-26","cited_by_count":226},{"id":"2301.07067","title":"Transformers as Algorithms: Generalization and Stability in In-context Learning","created":"2023-01-17","cited_by_count":222},{"id":"2310.16028","title":"What Algorithms can Transformers Learn? A Study in Length Generalization","created":"2023-10-24","cited_by_count":199},{"id":"2106.06981","title":"Thinking Like Transformers","created":"2021-06-13","cited_by_count":184},{"id":"2307.03576","title":"One Step of Gradient Descent is Provably the Optimal In-Context Learner with One Layer of Linear Self-Attention","created":"2023-07-07","cited_by_count":170},{"id":"2009.14286","title":"Benign overfitting in ridge regression","created":"2020-09-29","cited_by_count":153},{"id":"2309.14402","title":"Physics of Language Models: Part 3.2, Knowledge Manipulation","created":"2023-09-25","cited_by_count":143},{"id":"2402.14735","title":"How Transformers Learn Causal Structure with Gradient Descent","created":"2024-02-22","cited_by_count":118},{"id":"2310.05249","title":"In-Context Convergence of Transformers","created":"2023-10-08","cited_by_count":115},{"id":"2004.14786","title":"Perturbed Masking: Parameter-free Probing for Analyzing and Interpreting BERT","created":"2020-04-30","cited_by_count":109},{"id":"2301.05062","title":"Tracr: Compiled Transformers as a Laboratory for Interpretability","created":"2023-01-12","cited_by_count":104},{"id":"2305.16380","title":"Scan and Snap: Understanding Training Dynamics and Token Composition in 1-layer Transformer","created":"2023-05-25","cited_by_count":102},{"id":"2310.08391","title":"How Many Pretraining Tasks Are Needed for In-Context Learning of Linear Regression?","created":"2023-10-12","cited_by_count":96},{"id":"2303.04245","title":"How Do Transformers Learn Topic Structure: Towards a Mechanistic Understanding","created":"2023-03-07","cited_by_count":86},{"id":"2310.10616","title":"How Do Transformers Learn In-Context Beyond Simple Functions? A Case Study on Learning with Representations","created":"2023-10-16","cited_by_count":75},{"id":"2308.16898","title":"Transformers as Support Vector Machines","created":"2023-08-31","cited_by_count":74},{"id":"2402.19442","title":"Training Dynamics of Multi-Head Softmax Attention for In-Context Learning: Emergence, Convergence, and Optimality","created":"2024-02-29","cited_by_count":74},{"id":"2312.06528","title":"Transformers Implement Functional Gradient Descent to Learn Non-Linear Functions In Context","created":"2023-12-11","cited_by_count":63},{"id":"2307.14023","title":"Are Transformers with One Layer Self-Attention Using Low-Rank Weight Matrices Universal Approximators?","created":"2023-07-26","cited_by_count":47},{"id":"2306.01128","title":"Learning Transformer Programs","created":"2023-06-01","cited_by_count":41},{"id":"2306.02010","title":"Memorization Capacity of Multi-Head Attention in Transformers","created":"2023-06-03","cited_by_count":40},{"id":"2402.14951","title":"In-Context Learning of a Linear Transformer Block: Benefits of the MLP Component and One-Step GD Initialization","created":"2024-02-22","cited_by_count":33},{"id":"2108.04552","title":"The Benefits of Implicit Regularization from SGD in Least Squares Problems","created":"2021-08-10","cited_by_count":32},{"id":"2305.18699","title":"Approximation and Estimation Ability of Transformers for Sequence-to-Sequence Functions with Infinite Dimensional Input","created":"2023-05-30","cited_by_count":32},{"id":"2308.06912","title":"CausalLM is not optimal for in-context learning","created":"2023-08-14","cited_by_count":26},{"id":"2403.03183","title":"How Well Can Transformers Emulate In-context Newton's Method?","created":"2024-03-05","cited_by_count":25},{"id":"2404.01601","title":"What Can Transformer Learn with Varying Depth? Case Studies on Sequence Learning Tasks","created":"2024-04-02","cited_by_count":19},{"id":"2305.17040","title":"A Mechanism for Sample-Efficient In-Context Learning for Sparse Retrieval Tasks","created":"2023-05-26","cited_by_count":17},{"id":"2401.17426","title":"Superiority of Multi-Head Attention in In-Context Linear Regression","created":"2024-01-30","cited_by_count":17},{"id":"2104.09400","title":"Probing for Bridging Inference in Transformer Language Models","created":"2021-04-19","cited_by_count":14},{"id":"2203.03159","title":"Risk Bounds of Multi-Pass SGD for Least Squares in the Interpolation Regime","created":"2022-03-07","cited_by_count":8}],"next_cursor":null}