{"total":124,"rows":[{"id":"2403.19647","title":"Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Models","created":"2024-03-28","cited_by_count":435,"area":null},{"id":"2404.03592","title":"ReFT: Representation Finetuning for Language Models","created":"2024-04-04","cited_by_count":197,"area":null},{"id":"2402.11004","title":"The Evolution of Statistical Induction Heads: In-Context Learning Markov Chains","created":"2024-02-16","cited_by_count":118,"area":null},{"id":"2407.02646","title":"A Practical Review of Mechanistic Interpretability for Transformer-Based Language Models","created":"2024-07-02","cited_by_count":118,"area":"mechinterp"},{"id":"2406.11813","title":"How Do Large Language Models Acquire Factual Knowledge During Pretraining?","created":"2024-06-17","cited_by_count":90,"area":null},{"id":"2405.00208","title":"A Primer on the Inner Workings of Transformer-based Language Models","created":"2024-04-30","cited_by_count":85,"area":null},{"id":"2409.03752","title":"Attention Heads of Large Language Models: A Survey","created":"2024-09-05","cited_by_count":67,"area":null},{"id":"2407.10827","title":"LLM Circuit Analyses Are Consistent Across Training and Scale","created":"2024-07-15","cited_by_count":63,"area":null},{"id":"2407.17817","title":"Demystifying Verbatim Memorization in Large Language Models","created":"2024-07-25","cited_by_count":56,"area":null},{"id":"2506.03093","title":"From Flat to Hierarchical: Extracting Sparse Representations with Matching Pursuit","created":"2025-06-03","cited_by_count":49,"area":"mechinterp"},{"id":"2402.12560","title":"CausalGym: Benchmarking causal interpretability methods on linguistic tasks","created":"2024-02-19","cited_by_count":46,"area":null},{"id":"2501.16265","title":"Training Dynamics of In-Context Learning in Linear Attention","created":"2025-01-27","cited_by_count":46,"area":null},{"id":"2402.02364","title":"Loss Landscape Degeneracy and Stagewise Development in Transformers","created":"2024-02-04","cited_by_count":40,"area":null},{"id":"2410.02984","title":"Differentiation and Specialization of Attention Heads via the Refined Local Learning Coefficient","created":"2024-10-03","cited_by_count":40,"area":null},{"id":"2406.13229","title":"Probing the Emergence of Cross-lingual Alignment during LLM Training","created":"2024-06-19","cited_by_count":39,"area":null},{"id":"2503.02854","title":"(How) Do Language Models Track State?","created":"2025-03-04","cited_by_count":34,"area":null},{"id":"2406.14785","title":"Understanding Finetuning for Factual Knowledge Extraction","created":"2024-06-20","cited_by_count":32,"area":null},{"id":"2406.02128","title":"Iteration Head: A Mechanistic Study of Chain-of-Thought","created":"2024-06-04","cited_by_count":31,"area":"lr"},{"id":"2402.03902","title":"A phase transition between positional and semantic learning in a solvable model of dot-product attention","created":"2024-02-06","cited_by_count":30,"area":null},{"id":"2604.21691","title":"There Will Be a Scientific Theory of Deep Learning","created":"2026-04-23","cited_by_count":29,"area":null},{"id":"2410.09223","title":"The Same But Different: Structural Similarities and Differences in Multilingual Language Modeling","created":"2024-10-11","cited_by_count":28,"area":"mechinterp"},{"id":"2410.11840","title":"A Hitchhiker's Guide to Scaling Law Estimation","created":"2024-10-15","cited_by_count":27,"area":null},{"id":"2502.05489","title":"Mechanistic Interpretability of Emotion Inference in Large Language Models","created":"2025-02-08","cited_by_count":27,"area":"mechinterp"},{"id":"2407.12034","title":"Understanding Transformers via N-gram Statistics","created":"2024-06-30","cited_by_count":26,"area":null},{"id":"2410.09087","title":"Mechanistic?","created":"2024-10-07","cited_by_count":25,"area":"mechinterp"},{"id":"2404.16367","title":"Learning Syntax Without Planting Trees: Understanding Hierarchical Generalization in Transformers","created":"2024-04-25","cited_by_count":24,"area":null},{"id":"2408.01416","title":"The Quest for the Right Mediator: Surveying Mechanistic Interpretability Through the Lens of Causal Mediation Analysis","created":"2024-08-02","cited_by_count":24,"area":"mechinterp"},{"id":"2408.07144","title":"Language Models as Models of Language","created":"2024-08-13","cited_by_count":24,"area":null},{"id":"2310.16484","title":"Subspace Chronicles: How Linguistic Information Emerges, Shifts and Interacts during Language Model Training","created":"2023-10-25","cited_by_count":23,"area":null},{"id":"2311.07811","title":"In-context Learning Generalizes, But Not Always Robustly: The Case of Syntax","created":"2023-11-13","cited_by_count":22,"area":null},{"id":"2410.04368","title":"Algorithmic Capabilities of Random Transformers","created":"2024-10-06","cited_by_count":17,"area":null},{"id":"2410.01444","title":"Geometric Signatures of Compositionality Across a Language Model's Lifetime","created":"2024-10-02","cited_by_count":16,"area":null},{"id":"2508.12837","title":"Learning In-context n-grams with Transformers: Sub-n-grams Are Near-stationary Points","created":"2025-08-18","cited_by_count":16,"area":null},{"id":"2410.05464","title":"Progressive distillation induces an implicit curriculum","created":"2024-10-07","cited_by_count":15,"area":null},{"id":"2506.15872","title":"Hidden Breakthroughs in Language Model Training","created":"2025-06-18","cited_by_count":15,"area":null},{"id":"2603.10055","title":"Training Language Models via Neural Cellular Automata","created":"2026-03-09","cited_by_count":15,"area":null},{"id":"2310.12956","title":"Eureka-Moments in Transformers: Multi-Step Tasks Reveal Softmax Induced Optimization Problems","created":"2023-10-19","cited_by_count":14,"area":null},{"id":"2403.06925","title":"Transformers Learn Low Sensitivity Functions: Investigations and Implications","created":"2024-03-11","cited_by_count":14,"area":null},{"id":"2504.13292","title":"Let Me Grok for You: Accelerating Grokking via Embedding Transfer from a Weaker Model","created":"2025-04-17","cited_by_count":13,"area":null},{"id":"2510.24963","title":"Language Model Behavioral Phases are Consistent Across Architecture, Training Data, and Scale","created":"2025-10-28","cited_by_count":13,"area":null},{"id":"2410.03001","title":"Can Transformers Learn $n$-gram Language Models?","created":"2024-10-03","cited_by_count":11,"area":null},{"id":"2410.22244","title":"Abrupt Learning in Transformers: A Case Study on Matrix Completion","created":"2024-10-29","cited_by_count":11,"area":null},{"id":"2412.04619","title":"Sometimes I am a Tree: Data Drives Unstable Hierarchical Generalization","created":"2024-12-05","cited_by_count":10,"area":null},{"id":"2412.07646","title":"Searching for Structure: Investigating Emergent Communication with Large Language Models","created":"2024-12-10","cited_by_count":10,"area":null},{"id":"2502.18802","title":"Language Models Grow Less Humanlike beyond Phase Transition","created":"2025-02-26","cited_by_count":10,"area":null},{"id":"2503.04395","title":"Shaping Shared Languages: Human and Large Language Models' Inductive Biases in Emergent Communication","created":"2025-03-06","cited_by_count":10,"area":null},{"id":"2505.14824","title":"Tracing Multilingual Factual Knowledge Acquisition in Pretraining","created":"2025-05-20","cited_by_count":10,"area":null},{"id":"2601.21996","title":"Mechanistic Data Attribution: Tracing the Training Origins of Interpretable LLM Units","created":"2026-01-29","cited_by_count":10,"area":"mechinterp"},{"id":"2604.08510","title":"What do Language Models Learn and When? The Implicit Curriculum Hypothesis","created":"2026-04-09","cited_by_count":10,"area":null},{"id":"2605.29548","title":"Why Larger Models Learn More: Effects of Capacity, Interference, and Rare-Task Retention","created":"2026-05-28","cited_by_count":10,"area":null},{"id":"2402.13956","title":"Can You Learn Semantics Through Next-Word Prediction? The Case of Entailment","created":"2024-02-21","cited_by_count":9,"area":null},{"id":"2405.05966","title":"Natural Language Processing RELIES on Linguistics","created":"2024-05-09","cited_by_count":9,"area":null},{"id":"2408.06663","title":"Amuro and Char: Analyzing the Relationship between Pre-Training and Fine-Tuning of Large Language Models","created":"2024-08-13","cited_by_count":9,"area":null},{"id":"2405.15164","title":"From Frege to chatGPT: Compositionality in language, cognition, and deep neural networks","created":"2024-05-24","cited_by_count":7,"area":null},{"id":"2405.17088","title":"Phase Transitions in the Output Distribution of Large Language Models","created":"2024-05-27","cited_by_count":7,"area":null},{"id":"2405.17420","title":"Survival of the Fittest Representation: A Case Study with Modular Addition","created":"2024-05-27","cited_by_count":7,"area":null},{"id":"2502.17356","title":"Random Scaling of Emergent Capabilities","created":"2025-02-24","cited_by_count":7,"area":null},{"id":"2505.19548","title":"How Syntax Specialization Emerges in Language Models","created":"2025-05-26","cited_by_count":7,"area":null},{"id":"2506.11135","title":"Large Language Models and Emergence: A Complex Systems Perspective","created":"2025-06-10","cited_by_count":7,"area":null},{"id":"2506.13688","title":"What Happens During the Loss Plateau? Understanding Abrupt Learning in Transformers","created":"2025-06-16","cited_by_count":7,"area":null},{"id":"2511.16893","title":"Predicting the Emergence of Induction Heads in Language Model Pretraining","created":"2025-11-21","cited_by_count":7,"area":null},{"id":"2605.00842","title":"Understanding Emergent Misalignment via Feature Superposition Geometry","created":"2026-04-07","cited_by_count":7,"area":"mechinterp"},{"id":"2503.19260","title":"Linguistic Blind Spots of Large Language Models","created":"2025-03-25","cited_by_count":6,"area":null},{"id":"2509.05291","title":"Crosscoding Through Time: Tracking Emergence & Consolidation Of Linguistic Representations Throughout LLM Pretraining","created":"2025-09-05","cited_by_count":6,"area":null},{"id":"2509.22445","title":"Bridging Kolmogorov Complexity and Deep Learning: Asymptotically Optimal Description Length Objectives for Transformers","created":"2025-09-26","cited_by_count":6,"area":null},{"id":"2508.20015","title":"Decomposing Behavioral Phase Transitions in LLMs: Order Parameters for Emergent Misalignment","created":"2025-08-27","cited_by_count":5,"area":null},{"id":"2509.22831","title":"Toward a Theory of Generalizability in LLM Mechanistic Interpretability Research","created":"2025-09-26","cited_by_count":5,"area":"mechinterp"},{"id":"2512.16902","title":"In-Context Algebra","created":"2025-12-18","cited_by_count":5,"area":null},{"id":"2402.12691","title":"Tree-Planted Transformers: Unidirectional Transformer Language Models with Implicit Syntactic Supervision","created":"2024-02-20","cited_by_count":4,"area":null},{"id":"2411.07075","title":"Transformer verbatim in-context retrieval across time and scale","created":"2024-11-11","cited_by_count":4,"area":null},{"id":"2502.20779","title":"Triple Phase Transitions: Understanding the Learning Dynamics of Large Language Models from a Neuroscience Perspective","created":"2025-02-28","cited_by_count":4,"area":null},{"id":"2505.23489","title":"SGD as Free Energy Minimization: A Thermodynamic View on Neural Network Training","created":"2025-05-29","cited_by_count":4,"area":null},{"id":"2511.07368","title":"Distributional Biases in Post-Training: A Markovian Analysis of Reasoning Trajectories","created":"2025-11-10","cited_by_count":4,"area":"lr"},{"id":"2601.07524","title":"Stagewise Reinforcement Learning and the Geometry of the Regret Landscape","created":"2026-01-12","cited_by_count":4,"area":"deeprl"},{"id":"2407.01047","title":"Development of Cognitive Intelligence in Pre-trained Language Models","created":"2024-07-01","cited_by_count":3,"area":null},{"id":"2509.26643","title":"Convergence and Divergence of Language Models under Different Random Seeds","created":"2025-09-30","cited_by_count":3,"area":null},{"id":"2511.21974","title":"Start Making Sense(s): A Developmental Probe of Attention Specialization Using Lexical Ambiguity","created":"2025-11-26","cited_by_count":3,"area":null},{"id":"2601.03779","title":"Tracing the complexity profiles of different linguistic phenomena through the intrinsic dimension of LLM representations","created":"2026-01-07","cited_by_count":3,"area":null},{"id":"2601.21150","title":"Can Neural Networks Learn Small Algebraic Worlds? An Investigation Into the Group-theoretic Structures Learned By Narrow Models Trained To Predict Group Operations","created":"2026-01-29","cited_by_count":3,"area":null},{"id":"2606.06533","title":"Position: Don't Just \"Fix it in Post\": A Science of AI Must Study Training Dynamics","created":"2026-06-03","cited_by_count":3,"area":null},{"id":"2405.13231","title":"Multiple Realizability and the Rise of Deep Learning","created":"2024-05-21","cited_by_count":2,"area":null},{"id":"2409.13728","title":"Rule Extrapolation in Language Models: A Study of Compositional Generalization on OOD Prompts","created":"2024-09-09","cited_by_count":2,"area":null},{"id":"2409.17216","title":"Data-Centric AI Governance: Addressing the Limitations of Model-Focused Policies","created":"2024-09-25","cited_by_count":2,"area":null},{"id":"2509.12065","title":"Steering Language Models in Multi-Token Generation: A Case Study on Tense and Aspect","created":"2025-09-15","cited_by_count":2,"area":null},{"id":"2509.24198","title":"Negative Pre-activations Differentiate Syntax","created":"2025-09-29","cited_by_count":2,"area":null},{"id":"2510.07613","title":"Vocabulary embeddings organize linguistic structure early in language model training","created":"2025-10-08","cited_by_count":2,"area":null},{"id":"2601.18617","title":"Emergence of Phonemic, Syntactic, and Semantic Representations in Artificial Neural Networks","created":"2026-01-26","cited_by_count":2,"area":null},{"id":"2602.04081","title":"Abstraction Induces the Brain Alignment of Language and Speech Models","created":"2026-02-03","cited_by_count":2,"area":null},{"id":"2602.19143","title":"Incremental Learning of Sparse Attention Patterns in Transformers","created":"2026-02-22","cited_by_count":2,"area":null},{"id":"2603.03993","title":"Specialization of softmax attention heads: insights from the high-dimensional single-location model","created":"2026-03-04","cited_by_count":2,"area":null},{"id":"2603.17475","title":"Humans and transformer LMs: Abstraction drives language learning","created":"2026-03-18","cited_by_count":2,"area":null},{"id":"2603.26539","title":"How Open Must Language Models be to Enable Reliable Scientific Inference?","created":"2026-03-27","cited_by_count":2,"area":null},{"id":"2606.26749","title":"Structure Before Collapse: Transient semantic geometry in next-token prediction","created":"2026-06-25","cited_by_count":2,"area":null},{"id":"2402.13113","title":"When Only Time Will Tell: Interpreting How Transformers Process Local Ambiguities Through the Lens of Restart-Incrementality","created":"2024-02-20","cited_by_count":1,"area":null},{"id":"2411.05882","title":"When are 1.58 bits enough? A Bottom-up Exploration of BitNet Quantization","created":"2024-11-08","cited_by_count":1,"area":null},{"id":"2511.19328","title":"Understanding the Staged Dynamics of Transformers in Learning Latent Structure","created":"2025-11-24","cited_by_count":1,"area":null},{"id":"2601.03570","title":"How Do Large Language Models Learn Concepts During Continual Pre-Training?","created":"2026-01-07","cited_by_count":1,"area":null},{"id":"2601.19926","title":"The Grammar of Transformers: A Systematic Review of Interpretability Research on Syntactic Knowledge in Language Models","created":"2026-01-09","cited_by_count":1,"area":null},{"id":"2601.21191","title":"Function Words as Statistical Cues for Language Learning","created":"2026-01-29","cited_by_count":1,"area":null},{"id":"2604.02043","title":"Tracking the emergence of linguistic structure in self-supervised models learning from speech","created":"2026-04-02","cited_by_count":1,"area":null}],"next_cursor":"100"}