{"total":92,"rows":[{"id":"2402.11004","title":"The Evolution of Statistical Induction Heads: In-Context Learning Markov Chains","created":"2024-02-16","cited_by_count":118,"area":null},{"id":"2409.03752","title":"Attention Heads of Large Language Models: A Survey","created":"2024-09-05","cited_by_count":67,"area":null},{"id":"2502.01951","title":"On the Emergence of Position Bias in Transformers","created":"2025-02-04","cited_by_count":59,"area":null},{"id":"2410.13835","title":"Active-Dormant Attention Heads: Mechanistically Demystifying Extreme-Token Phenomena in LLMs","created":"2024-10-17","cited_by_count":56,"area":null},{"id":"2405.11751","title":"Asymptotic theory of in-context learning by linear attention","created":"2024-05-20","cited_by_count":55,"area":null},{"id":"2501.16265","title":"Training Dynamics of In-Context Learning in Linear Attention","created":"2025-01-27","cited_by_count":46,"area":null},{"id":"2405.19592","title":"Why Larger Language Models Do In-context Learning Differently?","created":"2024-05-30","cited_by_count":45,"area":null},{"id":"2406.09519","title":"Talking Heads: Understanding Inter-layer Communication in Transformer Language Models","created":"2024-06-13","cited_by_count":41,"area":null},{"id":"2505.13763","title":"Language Models Are Capable of Metacognitive Monitoring and Control of Their Internal Activations","created":"2025-05-19","cited_by_count":41,"area":null},{"id":"2503.21676","title":"How do language models learn facts? Dynamics, curricula and hallucinations","created":"2025-03-27","cited_by_count":37,"area":null},{"id":"2403.19521","title":"Interpreting Key Mechanisms of Factual Recall in Transformer-Based Language Models","created":"2024-03-28","cited_by_count":31,"area":null},{"id":"2402.03902","title":"A phase transition between positional and semantic learning in a solvable model of dot-product attention","created":"2024-02-06","cited_by_count":30,"area":null},{"id":"2604.21691","title":"There Will Be a Scientific Theory of Deep Learning","created":"2026-04-23","cited_by_count":29,"area":null},{"id":"2408.09503","title":"Out-of-distribution generalization via composition: a lens through induction heads in Transformers","created":"2024-08-18","cited_by_count":28,"area":null},{"id":"2408.12578","title":"A Percolation Model of Emergence: Analyzing Transformers Trained on a Formal Language","created":"2024-08-22","cited_by_count":26,"area":null},{"id":"2505.17863","title":"The emergence of sparse attention: impact of data distribution and benefits of repetition","created":"2025-05-23","cited_by_count":25,"area":null},{"id":"2410.11268","title":"Bypassing the Exponential Dependency: Looped Transformers Efficiently Learn In-context by Multi-step Gradient Descent","created":"2024-10-15","cited_by_count":22,"area":null},{"id":"2410.04691","title":"Deeper Insights Without Updates: The Power of In-Context Learning Over Fine-Tuning","created":"2024-10-07","cited_by_count":21,"area":null},{"id":"2412.00104","title":"Differential learning kinetics govern the transition from memorization to generalization during in-context learning","created":"2024-11-27","cited_by_count":21,"area":null},{"id":"2410.23042","title":"Toward Understanding In-context vs. In-weight Learning","created":"2024-10-30","cited_by_count":19,"area":null},{"id":"2505.11349","title":"Context parroting: A simple but tough-to-beat baseline for foundation models in scientific machine learning","created":"2025-05-16","cited_by_count":18,"area":null},{"id":"2503.05631","title":"Strategy Coopetition Explains the Emergence and Transience of In-Context Learning","created":"2025-03-07","cited_by_count":17,"area":null},{"id":"2402.01909","title":"On Catastrophic Inheritance of Large Foundation Models","created":"2024-02-02","cited_by_count":15,"area":null},{"id":"2405.14992","title":"Linking In-context Learning in Transformers to Human Episodic Memory","created":"2024-05-23","cited_by_count":15,"area":null},{"id":"2406.00053","title":"Dual Process Learning: Controlling Use of In-Context vs. In-Weights Strategies with Weight Forgetting","created":"2024-05-28","cited_by_count":15,"area":null},{"id":"2402.02212","title":"A Survey to Recent Progress Towards Understanding In-Context Learning","created":"2024-02-03","cited_by_count":14,"area":null},{"id":"2405.15618","title":"MLPs Learn In-Context on Regression and Classification Tasks","created":"2024-05-24","cited_by_count":13,"area":null},{"id":"2401.08309","title":"Anchor function: a type of benchmark functions for studying language models","created":"2024-01-16","cited_by_count":11,"area":null},{"id":"2505.16694","title":"Beyond Induction Heads: In-Context Meta Learning Induces Multi-Phase Circuit Emergence","created":"2025-05-22","cited_by_count":10,"area":null},{"id":"2505.18752","title":"Unifying Attention Heads and Task Vectors via Hidden State Geometry in In-Context Learning","created":"2025-05-24","cited_by_count":10,"area":null},{"id":"2510.06954","title":"From Condensation to Rank Collapse: A Two-Stage Analysis of Transformer Training Dynamics","created":"2025-10-08","cited_by_count":10,"area":null},{"id":"2405.15926","title":"Dissecting the Interplay of Attention Paths in a Statistical Mechanics Theory of Transformers","created":"2024-05-24","cited_by_count":9,"area":null},{"id":"2406.11890","title":"Unraveling the Mechanics of Learning-Based Demonstration Selection for In-Context Learning","created":"2024-06-14","cited_by_count":9,"area":null},{"id":"2410.11474","title":"How Transformers Get Rich: Approximation and Dynamics Analysis","created":"2024-10-15","cited_by_count":8,"area":null},{"id":"2505.05145","title":"Understanding In-context Learning of Addition via Activation Subspaces","created":"2025-05-08","cited_by_count":8,"area":null},{"id":"2509.26551","title":"Pretrain-Test Task Alignment Governs Generalization in In-Context Learning","created":"2025-09-30","cited_by_count":8,"area":null},{"id":"2405.18202","title":"IM-Context: In-Context Learning for Imbalanced Regression Tasks","created":"2024-05-28","cited_by_count":7,"area":null},{"id":"2411.12118","title":"Mechanism and Emergence of Stacked Attention Heads in Multi-Layer Transformers","created":"2024-11-18","cited_by_count":7,"area":null},{"id":"2501.18187","title":"On the Role of Transformer Feed-Forward Layers in Nonlinear In-Context Learning","created":"2025-01-30","cited_by_count":7,"area":null},{"id":"2502.05164","title":"In-context denoising with one-layer transformers: connections between attention and associative memory retrieval","created":"2025-02-07","cited_by_count":7,"area":null},{"id":"2505.12138","title":"Transformer learns the cross-task prior and regularization for in-context learning","created":"2025-05-17","cited_by_count":7,"area":null},{"id":"2506.13688","title":"What Happens During the Loss Plateau? Understanding Abrupt Learning in Transformers","created":"2025-06-16","cited_by_count":7,"area":null},{"id":"2511.16893","title":"Predicting the Emergence of Induction Heads in Language Model Pretraining","created":"2025-11-21","cited_by_count":7,"area":null},{"id":"2512.18634","title":"From Shortcut to Induction Head: How Data Diversity Shapes Algorithm Selection in Transformers","created":"2025-12-21","cited_by_count":6,"area":null},{"id":"2410.05448","title":"Task Diversity Shortens the ICL Plateau","created":"2024-10-07","cited_by_count":5,"area":null},{"id":"2411.05943","title":"Quantifying artificial intelligence through algorithmic generalization","created":"2024-11-08","cited_by_count":5,"area":null},{"id":"2511.01033","title":"On the Emergence of Induction Heads for In-Context Learning","created":"2025-11-02","cited_by_count":5,"area":null},{"id":"2501.06256","title":"Unlocking In-Context Learning for Natural Datasets Beyond Language Modelling","created":"2025-01-09","cited_by_count":4,"area":null},{"id":"2504.19395","title":"ICL CIPHERS: Quantifying \"Learning\" in In-Context Learning via Substitution Ciphers","created":"2025-04-28","cited_by_count":4,"area":null},{"id":"2510.10425","title":"Softmax $\\geq$ Linear: Transformers may learn to classify in-context by kernel gradient descent","created":"2025-10-12","cited_by_count":4,"area":null},{"id":"2601.16407","title":"Jacobian Scopes: token-level causal attributions in LLMs","created":"2026-01-23","cited_by_count":4,"area":null},{"id":"2604.12151","title":"Distinct mechanisms underlying in-context learning in transformers","created":"2026-04-14","cited_by_count":4,"area":null},{"id":"2406.02663","title":"Demystifying Spectral Bias on Real-World Data","created":"2024-06-04","cited_by_count":3,"area":null},{"id":"2410.05493","title":"An Information-Theoretic Approach to Understanding Transformers' In-Context Learning of Variable-Order Markov Chains","created":"2024-10-07","cited_by_count":3,"area":null},{"id":"2412.15113","title":"Associative memory inspires improvements for in-context learning using a novel attention residual stream architecture","created":"2024-12-19","cited_by_count":3,"area":null},{"id":"2509.09232","title":"Medverse: A Universal Model for Full-Resolution 3D Medical Image Segmentation, Transformation and Enhancement","created":"2025-09-11","cited_by_count":3,"area":null},{"id":"2405.15115","title":"Towards Better Understanding of In-Context Learning Ability from In-Context Uncertainty Quantification","created":"2024-05-24","cited_by_count":2,"area":null},{"id":"2602.00533","title":"Convergent World Representations and Divergent Tasks","created":"2026-01-31","cited_by_count":2,"area":null},{"id":"2602.18679","title":"Transformers for dynamical systems learn transfer operators in-context","created":"2026-02-21","cited_by_count":2,"area":null},{"id":"2602.19143","title":"Incremental Learning of Sparse Attention Patterns in Transformers","created":"2026-02-22","cited_by_count":2,"area":null},{"id":"2607.07678","title":"How Data Shapes RoPE Frequency Usage: From Positional Scale Matching to Length Generalization","created":"2026-07-08","cited_by_count":2,"area":null},{"id":"2608.17957","title":"Understanding the Surprising Generalization Properties of Tabular Foundation Models","created":"2026-08-18","cited_by_count":2,"area":null},{"id":"2407.10949","title":"Representing Rule-based Chatbots with Transformers","created":"2024-07-15","cited_by_count":1,"area":null},{"id":"2410.04560","title":"GAMformer: Bridging Tabular Foundation Models and Interpretable Machine Learning","created":"2024-10-06","cited_by_count":1,"area":null},{"id":"2410.09701","title":"Transformers as Game Players: Provable In-context Game-playing Capabilities of Pre-trained Models","created":"2024-10-13","cited_by_count":1,"area":null},{"id":"2410.24050","title":"A Mechanistic Study of Transformers Training Dynamics","created":"2024-10-31","cited_by_count":1,"area":null},{"id":"2501.02378","title":"A ghost mechanism: An analytical model of abrupt learning in recurrent networks","created":"2025-01-04","cited_by_count":1,"area":null},{"id":"2504.17068","title":"In-Context Learning can distort the relationship between sequence likelihoods and biological fitness","created":"2025-04-23","cited_by_count":1,"area":null},{"id":"2509.21042","title":"LayerNorm Induces Recency Bias in Transformer Decoders","created":"2025-09-25","cited_by_count":1,"area":null},{"id":"2511.12797","title":"Genomic Next-Token Predictors are In-Context Learners","created":"2025-11-16","cited_by_count":1,"area":null},{"id":"2511.19328","title":"Understanding the Staged Dynamics of Transformers in Learning Latent Structure","created":"2025-11-24","cited_by_count":1,"area":null},{"id":"2604.23267","title":"Fine-tuning vs. In-context Learning in Large Language Models: A Formal Language Learning Perspective","created":"2026-04-25","cited_by_count":1,"area":null},{"id":"2607.03660","title":"Sequential Correlations Change In-Context Learning: Effective Context Length and Architectural Mismatch","created":"2026-07-04","cited_by_count":1,"area":null},{"id":"2609.14011","title":"Convergent Emergence of In-Context Learning Across Modalities","created":"2026-09-12","cited_by_count":1,"area":null},{"id":"2609.36012","title":"In-Context Learning for Robots: Methods and Applications","created":"2026-09-28","cited_by_count":1,"area":null},{"id":"2505.14233","title":"Mechanistic Fine-tuning for In-context Learning","created":"2025-05-20","cited_by_count":0,"area":null},{"id":"2507.01414","title":"Decomposing Prediction Mechanisms for In-Context Recall","created":"2025-07-02","cited_by_count":0,"area":null},{"id":"2511.05743","title":"In-Context Learning Without Copying","created":"2025-11-07","cited_by_count":0,"area":null},{"id":"2602.07562","title":"Gaussian Match-and-Copy: A Minimalist Benchmark for Studying Transformer Induction","created":"2026-02-07","cited_by_count":0,"area":null},{"id":"2603.13459","title":"Reconciling In-Context and In-Weight Learning via Dual Representation Space Encoding","created":"2026-03-13","cited_by_count":0,"area":null},{"id":"2604.11613","title":"Symmetry Reveals Layerwise Dynamics: How Transformers Perform In-Context Classification","created":"2026-04-13","cited_by_count":0,"area":null},{"id":"2605.20299","title":"Mechanisms of Misgeneralization in Physical Sequence Modeling","created":"2026-05-19","cited_by_count":0,"area":null},{"id":"2605.26489","title":"The Stability of Singular Distribution: A Spectral Perspective on the Two-Phase Dynamics of Language Model Pre-training","created":"2026-05-26","cited_by_count":0,"area":null},{"id":"2605.31497","title":"Assign and Add: A Mechanistic Study of Compositional Arithmetic","created":"2026-05-29","cited_by_count":0,"area":null},{"id":"2606.16694","title":"When Does Depth Matter For In-Context Learning? Adaptive Inference in Deep Transformers","created":"2026-06-15","cited_by_count":0,"area":null},{"id":"2607.04846","title":"Pretraining Curricula Enable Selective Fine-tuning","created":"2026-07-06","cited_by_count":0,"area":null},{"id":"2607.06621","title":"Fingerprint, Not Blueprint: How Positional Schemes Set the Default Spectral Algebra of Attention","created":"2026-07-07","cited_by_count":0,"area":null},{"id":"2608.24460","title":"Shortcut Before Circuit: Document Statistics Time In-Context Conflict Resolution","created":"2026-08-25","cited_by_count":0,"area":null},{"id":"2609.09306","title":"Gradland: On Phenomenal Experience, Differentiated Across Many Dimensions","created":"2026-09-08","cited_by_count":0,"area":null},{"id":"2609.16540","title":"On the Importance of Gating: Memorization vs. In-Context Learning in State Space Models","created":"2026-09-15","cited_by_count":0,"area":null},{"id":"2610.12571","title":"An all-transistor physical-learning transformer can be trained to perform in-context learning","created":"2026-10-08","cited_by_count":0,"area":null},{"id":"2610.19113","title":"From Copying to Composition: How Language Models Develop In-Context Learning","created":"2026-10-13","cited_by_count":0,"area":null}],"next_cursor":null}