{"list":"other","total":71,"rows":[{"id":"1706.03762","title":"Attention Is All You Need","created":"2017-06-12","cited_by_count":60361},{"id":"2005.14165","title":"Language Models are Few-Shot Learners","created":"2020-05-28","cited_by_count":32419},{"id":"1912.01703","title":"PyTorch: An Imperative Style, High-Performance Deep Learning Library","created":"2019-12-03","cited_by_count":20305},{"id":"2104.09864","title":"RoFormer: Enhanced Transformer with Rotary Position Embedding","created":"2021-04-20","cited_by_count":4703},{"id":"1310.4546","title":"Distributed Representations of Words and Phrases and their Compositionality","created":"2013-10-16","cited_by_count":3474},{"id":"2202.05262","title":"Locating and Editing Factual Associations in GPT","created":"2022-02-10","cited_by_count":2691},{"id":"1610.01644","title":"Understanding intermediate layers using linear classifier probes","created":"2016-10-05","cited_by_count":1353},{"id":"2211.00593","title":"Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small","created":"2022-11-01","cited_by_count":1165},{"id":"2111.02080","title":"An Explanation of In-context Learning as Implicit Bayesian Inference","created":"2021-11-03","cited_by_count":938},{"id":"2102.12452","title":"Probing Classifiers: Promises, Shortcomings, and Advances","created":"2021-02-24","cited_by_count":876},{"id":"2301.05217","title":"Progress measures for grokking via mechanistic interpretability","created":"2023-01-12","cited_by_count":839},{"id":"2212.07677","title":"Transformers learn in-context by gradient descent","created":"2022-12-15","cited_by_count":732},{"id":"2211.15661","title":"What learning algorithm is in-context learning? Investigations with linear models","created":"2022-11-28","cited_by_count":684},{"id":"2201.02177","title":"Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets","created":"2022-01-06","cited_by_count":629},{"id":"2410.05229","title":"GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Models","created":"2024-10-07","cited_by_count":550},{"id":"2210.13382","title":"Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Task","created":"2022-10-24","cited_by_count":431},{"id":"2310.15916","title":"In-Context Learning Creates Task Vectors","created":"2023-10-24","cited_by_count":312},{"id":"2310.15213","title":"Function Vectors in Large Language Models","created":"2023-10-23","cited_by_count":308},{"id":"1301.2300","title":"Direct and Indirect Effects","created":"2013-01-10","cited_by_count":280},{"id":"2210.10749","title":"Transformers Learn Shortcuts to Automata","created":"2022-10-19","cited_by_count":268},{"id":"2205.10343","title":"Towards Understanding Grokking: An Effective Theory of Representation Learning","created":"2022-05-20","cited_by_count":230},{"id":"2303.02536","title":"Finding Alignments Between Interpretable Causal Variables and Distributed Neural Representations","created":"2023-03-05","cited_by_count":209},{"id":"2301.04709","title":"Causal Abstraction: A Theoretical Foundation for Mechanistic Interpretability","created":"2023-01-11","cited_by_count":205},{"id":"2306.17844","title":"The Clock and the Pizza: Two Stories in Mechanistic Explanation of Neural Networks","created":"2023-06-30","cited_by_count":156},{"id":"2402.14811","title":"Fine-Tuning Enhances Existing Mechanisms: A Case Study on Entity Tracking","created":"2024-02-22","cited_by_count":149},{"id":"2306.15063","title":"Pretraining task diversity and the emergence of non-Bayesian in-context learning for regression","created":"2023-06-26","cited_by_count":146},{"id":"2209.02535","title":"Analyzing Transformers in Embedding Space","created":"2022-09-06","cited_by_count":132},{"id":"2309.07311","title":"Sudden Drops in the Loss: Syntax Acquisition, Phase Transitions, and Simplicity Bias in MLMs","created":"2023-09-13","cited_by_count":124},{"id":"2310.17191","title":"How do Language Models Bind Entities in Context?","created":"2023-10-26","cited_by_count":108},{"id":"2407.20311","title":"Physics of Language Models: Part 2.1, Grade-School Math and the Hidden Reasoning Process","created":"2024-07-29","cited_by_count":108},{"id":"2206.04301","title":"Unveiling Transformers with LEGO: a synthetic reasoning task","created":"2022-06-09","cited_by_count":106},{"id":"2212.04458","title":"General-Purpose In-Context Learning by Meta-Learning Transformers","created":"2022-12-08","cited_by_count":105},{"id":"2305.08809","title":"Interpretability at Scale: Identifying Causal Mechanisms in Alpaca","created":"2023-05-15","cited_by_count":103},{"id":"2410.21272","title":"Arithmetic Without Algorithms: Language Models Solve Math With a Bag of Heuristics","created":"2024-10-28","cited_by_count":97},{"id":"2401.12973","title":"In-Context Language Learning: Architectures and Algorithms","created":"2024-01-23","cited_by_count":89},{"id":"2404.07129","title":"What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation","created":"2024-04-10","cited_by_count":89},{"id":"2208.02957","title":"Meaning without reference in large language models","created":"2022-08-05","cited_by_count":76},{"id":"2502.00873","title":"Language Models Use Trigonometry to Do Addition","created":"2025-02-02","cited_by_count":70},{"id":"2502.14010","title":"Which Attention Heads Matter for In-Context Learning?","created":"2025-02-19","cited_by_count":61},{"id":"2412.01003","title":"Competition Dynamics Shape Algorithmic Phases of In-Context Learning","created":"2024-12-01","cited_by_count":54},{"id":"2501.00070","title":"ICLR: In-Context Learning of Representations","created":"2024-12-29","cited_by_count":50},{"id":"2402.11917","title":"A Mechanistic Analysis of a Transformer Trained on a Symbolic Multi-Step Reasoning Task","created":"2024-02-19","cited_by_count":49},{"id":"2402.13055","title":"Identifying Semantic Induction Heads to Understand In-Context Learning","created":"2024-02-20","cited_by_count":47},{"id":"2407.14561","title":"NNsight and NDIF: Democratizing Access to Open-Weight Foundation Model Internals","created":"2024-07-18","cited_by_count":45},{"id":"2406.09519","title":"Talking Heads: Understanding Inter-layer Communication in Transformer Language Models","created":"2024-06-13","cited_by_count":41},{"id":"2402.02364","title":"Loss Landscape Degeneracy and Stagewise Development in Transformers","created":"2024-02-04","cited_by_count":40},{"id":"2406.02550","title":"Learning to grok: Emergence of in-context learning and skill composition in modular arithmetic tasks","created":"2024-06-04","cited_by_count":40},{"id":"2406.03445","title":"Pre-trained Large Language Models Use Fourier Features to Compute Addition","created":"2024-06-05","cited_by_count":39},{"id":"2505.14685","title":"Language Models use Lookbacks to Track Beliefs","created":"2025-05-20","cited_by_count":39},{"id":"2412.03782","title":"The broader spectrum of in-context learning","created":"2024-12-05","cited_by_count":35},{"id":"2503.02854","title":"(How) Do Language Models Track State?","created":"2025-03-04","cited_by_count":34},{"id":"2311.07568","title":"Feature emergence via margin maximization: case studies in algebraic tasks","created":"2023-11-13","cited_by_count":33},{"id":"2502.20332","title":"Emergent Symbolic Mechanisms Support Abstract Reasoning in Large Language Models","created":"2025-02-27","cited_by_count":31},{"id":"2409.05448","title":"Representational Analysis of Binding in Language Models","created":"2024-09-09","cited_by_count":28},{"id":"2505.17863","title":"The emergence of sparse attention: impact of data distribution and benefits of repetition","created":"2025-05-23","cited_by_count":25},{"id":"2312.06581","title":"Grokking Group Multiplication with Cosets","created":"2023-12-11","cited_by_count":24},{"id":"2408.01416","title":"The Quest for the Right Mediator: Surveying Mechanistic Interpretability Through the Lens of Causal Mediation Analysis","created":"2024-08-02","cited_by_count":24},{"id":"2504.03022","title":"The Dual-Route Model of Induction","created":"2025-04-03","cited_by_count":24},{"id":"2505.20896","title":"How Do Transformers Learn Variable Binding in Symbolic Programs?","created":"2025-05-27","cited_by_count":21},{"id":"2412.04703","title":"Transformers Struggle to Learn to Search","created":"2024-12-06","cited_by_count":19},{"id":"2410.04368","title":"Algorithmic Capabilities of Random Transformers","created":"2024-10-06","cited_by_count":17},{"id":"2506.17859","title":"In-Context Learning Strategies Emerge Rationally","created":"2025-06-21","cited_by_count":16},{"id":"2502.01540","title":"What is a Number, That a Large Language Model May Know It?","created":"2025-02-03","cited_by_count":15},{"id":"2506.15872","title":"Hidden Breakthroughs in Language Model Training","created":"2025-06-18","cited_by_count":15},{"id":"2307.01201","title":"Schema-learning and rebinding as mechanisms of in-context learning and emergence","created":"2023-06-16","cited_by_count":14},{"id":"2410.15580","title":"Language Models are Symbolic Learners in Arithmetic","created":"2024-10-21","cited_by_count":11},{"id":"2505.16694","title":"Beyond Induction Heads: In-Context Meta Learning Induces Multi-Phase Circuit Emergence","created":"2025-05-22","cited_by_count":10},{"id":"2505.05145","title":"Understanding In-context Learning of Addition via Activation Subspaces","created":"2025-05-08","cited_by_count":8},{"id":"2505.23701","title":"Can LLMs Reason Abstractly Over Math Word Problems Without CoT? Disentangling Abstract Formulation From Arithmetic Computation","created":"2025-05-29","cited_by_count":8},{"id":"2510.00184","title":"Why Can't Transformers Learn Multiplication? Reverse-Engineering Reveals Long-Range Dependency Pitfalls","created":"2025-09-30","cited_by_count":7},{"id":"2410.05448","title":"Task Diversity Shortens the ICL Plateau","created":"2024-10-07","cited_by_count":5}],"next_cursor":null}