{"list":"other","total":92,"rows":[{"id":"1706.03762","title":"Attention Is All You Need","created":"2017-06-12","cited_by_count":60361},{"id":"1810.04805","title":"BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding","created":"2018-10-11","cited_by_count":40399},{"id":"2005.14165","title":"Language Models are Few-Shot Learners","created":"2020-05-28","cited_by_count":32419},{"id":"2303.08774","title":"GPT-4 Technical Report","created":"2023-03-15","cited_by_count":18396},{"id":"2203.02155","title":"Training language models to follow instructions with human feedback","created":"2022-03-04","cited_by_count":15971},{"id":"2407.21783","title":"The Llama 3 Herd of Models","created":"2024-07-31","cited_by_count":14715},{"id":"2302.13971","title":"LLaMA: Open and Efficient Foundation Language Models","created":"2023-02-27","cited_by_count":14132},{"id":"1907.11692","title":"RoBERTa: A Robustly Optimized BERT Pretraining Approach","created":"2019-07-26","cited_by_count":13372},{"id":"2307.09288","title":"Llama 2: Open Foundation and Fine-Tuned Chat Models","created":"2023-07-18","cited_by_count":13028},{"id":"2201.11903","title":"Chain-of-Thought Prompting Elicits Reasoning in Large Language Models","created":"2022-01-28","cited_by_count":12459},{"id":"2110.14168","title":"Training Verifiers to Solve Math Word Problems","created":"2021-10-27","cited_by_count":10109},{"id":"2104.09864","title":"RoFormer: Enhanced Transformer with Rotary Position Embedding","created":"2021-04-20","cited_by_count":4703},{"id":"1409.0473","title":"Neural Machine Translation by Jointly Learning to Align and Translate","created":"2014-09-01","cited_by_count":4548},{"id":"1301.3781","title":"Efficient Estimation of Word Representations in Vector Space","created":"2013-01-16","cited_by_count":4375},{"id":"2205.11916","title":"Large Language Models are Zero-Shot Reasoners","created":"2022-05-24","cited_by_count":4374},{"id":"2310.06825","title":"Mistral 7B","created":"2023-10-10","cited_by_count":3164},{"id":"2303.12712","title":"Sparks of Artificial General Intelligence: Early experiments with GPT-4","created":"2023-03-22","cited_by_count":2592},{"id":"2206.07682","title":"Emergent Abilities of Large Language Models","created":"2022-06-15","cited_by_count":2512},{"id":"2408.00118","title":"Gemma 2: Improving Open Language Models at a Practical Size","created":"2024-07-31","cited_by_count":1998},{"id":"2304.01373","title":"Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling","created":"2023-04-03","cited_by_count":1786},{"id":"2202.12837","title":"Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?","created":"2022-02-25","cited_by_count":1434},{"id":"2102.09690","title":"Calibrate Before Use: Improving Few-Shot Performance of Language Models","created":"2021-02-19","cited_by_count":1392},{"id":"2012.14913","title":"Transformer Feed-Forward Layers Are Key-Value Memories","created":"2020-12-29","cited_by_count":1336},{"id":"2306.03341","title":"Inference-Time Intervention: Eliciting Truthful Answers from a Language Model","created":"2023-06-06","cited_by_count":1302},{"id":"2211.00593","title":"Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small","created":"2022-11-01","cited_by_count":1165},{"id":"2104.08786","title":"Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity","created":"2021-04-18","cited_by_count":1135},{"id":"2212.04089","title":"Editing Models with Task Arithmetic","created":"2022-12-08","cited_by_count":1120},{"id":"2209.10652","title":"Toy Models of Superposition","created":"2022-09-21","cited_by_count":971},{"id":"2209.11895","title":"In-context Learning and Induction Heads","created":"2022-09-24","cited_by_count":964},{"id":"2111.02080","title":"An Explanation of In-context Learning as Implicit Bayesian Inference","created":"2021-11-03","cited_by_count":938},{"id":"2403.08295","title":"Gemma: Open Models Based on Gemini Research and Technology","created":"2024-03-13","cited_by_count":916},{"id":"1411.4555","title":"Show and Tell: A Neural Image Caption Generator","created":"2014-11-17","cited_by_count":879},{"id":"2301.05217","title":"Progress measures for grokking via mechanistic interpretability","created":"2023-01-12","cited_by_count":839},{"id":"2212.07677","title":"Transformers learn in-context by gradient descent","created":"2022-12-15","cited_by_count":732},{"id":"2311.03658","title":"The Linear Representation Hypothesis and the Geometry of Large Language Models","created":"2023-11-07","cited_by_count":716},{"id":"2208.01066","title":"What Can Transformers Learn In-Context? A Case Study of Simple Function Classes","created":"2022-08-01","cited_by_count":715},{"id":"2211.15661","title":"What learning algorithm is in-context learning? Investigations with linear models","created":"2022-11-28","cited_by_count":684},{"id":"2201.02177","title":"Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets","created":"2022-01-06","cited_by_count":629},{"id":"2402.00838","title":"OLMo: Accelerating the Science of Language Models","created":"2024-02-01","cited_by_count":613},{"id":"2102.07350","title":"Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigm","created":"2021-02-15","cited_by_count":606},{"id":"2406.04093","title":"Scaling and evaluating sparse autoencoders","created":"2024-06-06","cited_by_count":553},{"id":"2311.16867","title":"The Falcon Series of Open Language Models","created":"2023-11-28","cited_by_count":535},{"id":"2305.18654","title":"Faith and Fate: Limits of Transformers on Compositionality","created":"2023-05-29","cited_by_count":503},{"id":"2110.15943","title":"MetaICL: Learning to Learn In Context","created":"2021-10-29","cited_by_count":454},{"id":"2306.09927","title":"Trained Transformers Learn Linear Models In-Context","created":"2023-06-16","cited_by_count":337},{"id":"2305.19466","title":"The Impact of Positional Encoding on Length Generalization in Transformers","created":"2023-05-31","cited_by_count":327},{"id":"2304.01933","title":"LLM-Adapters: An Adapter Family for Parameter-Efficient Fine-Tuning of Large Language Models","created":"2023-04-04","cited_by_count":324},{"id":"2306.04637","title":"Transformers as Statisticians: Provable In-Context Learning with In-Context Algorithm Selection","created":"2023-06-07","cited_by_count":298},{"id":"2104.08315","title":"Surface Form Competition: Why the Highest Probability Answer Isn't Always Right","created":"2021-04-16","cited_by_count":263},{"id":"2307.02477","title":"Reasoning or Reciting? Exploring the Capabilities and Limitations of Language Models Through Counterfactual Tasks","created":"2023-07-05","cited_by_count":246},{"id":"2207.04901","title":"Exploring Length Generalization in Large Language Models","created":"2022-07-11","cited_by_count":206},{"id":"2404.11018","title":"Many-Shot In-Context Learning","created":"2024-04-17","cited_by_count":206},{"id":"2310.16028","title":"What Algorithms can Transformers Learn? A Study in Length Generalization","created":"2023-10-24","cited_by_count":199},{"id":"2404.09932","title":"Foundational Challenges in Assuring Alignment and Safety of Large Language Models","created":"2024-04-15","cited_by_count":199},{"id":"2307.03576","title":"One Step of Gradient Descent is Provably the Optimal In-Context Learner with One Layer of Linear Self-Attention","created":"2023-07-07","cited_by_count":170},{"id":"1601.03764","title":"Linear Algebraic Structure of Word Senses, with Applications to Polysemy","created":"2016-01-14","cited_by_count":165},{"id":"2110.07814","title":"Meta-learning via Language Model In-context Tuning","created":"2021-10-15","cited_by_count":164},{"id":"2306.17844","title":"The Clock and the Pizza: Two Stories in Mechanistic Explanation of Neural Networks","created":"2023-06-30","cited_by_count":156},{"id":"2305.09731","title":"What In-Context Learning \"Learns\" In-Context: Disentangling Task Recognition and Task Learning","created":"2023-05-16","cited_by_count":152},{"id":"2210.01117","title":"Omnigrok: Grokking Beyond Algorithmic Data","created":"2022-10-03","cited_by_count":146},{"id":"2307.03381","title":"Teaching Arithmetic to Small Transformers","created":"2023-07-07","cited_by_count":123},{"id":"2312.03002","title":"The mechanistic basis of data dependence and abrupt learning in an in-context classification task","created":"2023-12-03","cited_by_count":118},{"id":"2402.14735","title":"How Transformers Learn Causal Structure with Gradient Descent","created":"2024-02-22","cited_by_count":118},{"id":"2303.02861","title":"Multitask Prompt Tuning Enables Parameter-Efficient Transfer Learning","created":"2023-03-06","cited_by_count":115},{"id":"2307.15936","title":"A Theory for Emergence of Complex Skills in Language Models","created":"2023-07-29","cited_by_count":113},{"id":"2103.15949","title":"Transformer visualization via dictionary learning: contextualized embedding as a linear superposition of transformer factors","created":"2021-03-29","cited_by_count":108},{"id":"2310.08744","title":"Circuit Component Reuse Across Tasks in Transformer Language Models","created":"2023-10-12","cited_by_count":108},{"id":"2206.04301","title":"Unveiling Transformers with LEGO: a synthetic reasoning task","created":"2022-06-09","cited_by_count":106},{"id":"2401.12973","title":"In-Context Language Learning: Architectures and Algorithms","created":"2024-01-23","cited_by_count":89},{"id":"2404.07129","title":"What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation","created":"2024-04-10","cited_by_count":89},{"id":"2303.07971","title":"A Theory of Emergent In-Context Learning as Implicit Structure Induction","created":"2023-03-14","cited_by_count":86},{"id":"2405.00208","title":"A Primer on the Inner Workings of Transformer-based Language Models","created":"2024-04-30","cited_by_count":85},{"id":"2305.15269","title":"Testing the General Deductive Reasoning Capacity of Large Language Models Using OOD Examples","created":"2023-05-24","cited_by_count":84},{"id":"2307.09476","title":"Overthinking the Truth: Understanding how Language Models Process False Demonstrations","created":"2023-07-18","cited_by_count":81},{"id":"2311.18817","title":"Dichotomy of Early and Late Phase Implicit Biases Can Provably Induce Grokking","created":"2023-11-30","cited_by_count":80},{"id":"2405.15071","title":"Grokked Transformers are Implicit Reasoners: A Mechanistic Journey to the Edge of Generalization","created":"2024-05-23","cited_by_count":79},{"id":"2312.09230","title":"Successor Heads: Recurring, Interpretable Attention Heads In The Wild","created":"2023-12-14","cited_by_count":77},{"id":"2301.13105","title":"Generalization on the Unseen, Logic Reasoning and Degree Curriculum","created":"2023-01-30","cited_by_count":73},{"id":"2305.14825","title":"Large Language Models are In-Context Semantic Reasoners rather than Symbolic Reasoners","created":"2023-05-24","cited_by_count":63},{"id":"2405.19592","title":"Why Larger Language Models Do In-context Learning Differently?","created":"2024-05-30","cited_by_count":45},{"id":"2407.15720","title":"Do Large Language Models Have Compositional Ability? An Investigation into Limitations and Scalability","created":"2024-07-22","cited_by_count":40},{"id":"2405.05219","title":"Conv-Basis: A New Paradigm for Efficient Attention Inference and Gradient Computation in Transformers","created":"2024-05-08","cited_by_count":32},{"id":"2402.15017","title":"Towards Few-Shot Adaptation of Foundation Models via Multitask Finetuning","created":"2024-02-22","cited_by_count":31},{"id":"2407.20199","title":"Emergence in non-neural models: grokking modular arithmetic via average gradient outer product","created":"2024-07-29","cited_by_count":28},{"id":"2310.09753","title":"When can transformers reason with abstract symbols?","created":"2023-10-15","cited_by_count":24},{"id":"2405.01964","title":"Position: Understanding LLMs Requires More Than Statistical Generalization","created":"2024-05-03","cited_by_count":21},{"id":"2403.04571","title":"Machine learning and information theory concepts towards an AI Mathematician","created":"2024-03-07","cited_by_count":20},{"id":"2404.01601","title":"What Can Transformer Learn with Varying Depth? Case Studies on Sequence Learning Tasks","created":"2024-04-02","cited_by_count":19},{"id":"2310.04861","title":"Uncovering hidden geometry in Transformers via disentangling position and context","created":"2023-10-07","cited_by_count":18},{"id":"2405.15302","title":"Understanding the Language Model to Solve the Symbolic Multi-Step Reasoning Problem from the Perspective of Buffer Mechanism","created":"2024-05-24","cited_by_count":15},{"id":"2407.14885","title":"Falcon2-11B Technical Report","created":"2024-07-20","cited_by_count":12},{"id":"2302.07866","title":"Do Deep Neural Networks Capture Compositionality in Arithmetic Reasoning?","created":"2023-02-15","cited_by_count":11}],"next_cursor":null}