{"total":106,"rows":[{"id":"2307.05782","title":"Large Language Models","created":"2023-07-11","cited_by_count":780,"area":null},{"id":"2301.13196","title":"Looped Transformers as Programmable Computers","created":"2023-01-30","cited_by_count":237,"area":null},{"id":"2402.01032","title":"Repeat After Me: Transformers are Better than State Space Models at Copying","created":"2024-02-01","cited_by_count":216,"area":null},{"id":"2310.16028","title":"What Algorithms can Transformers Learn? A Study in Length Generalization","created":"2023-10-24","cited_by_count":199,"area":null},{"id":"2207.00729","title":"The Parallelism Tradeoff: Limitations of Log-Precision Transformers","created":"2022-07-02","cited_by_count":193,"area":null},{"id":"2306.00802","title":"Birth of a Transformer: A Memory Viewpoint","created":"2023-06-01","cited_by_count":161,"area":null},{"id":"2311.00208","title":"What Formal Languages Can Transformers Express? A Survey","created":"2023-11-01","cited_by_count":124,"area":null},{"id":"2301.05062","title":"Tracr: Compiled Transformers as a Laboratory for Interpretability","created":"2023-01-12","cited_by_count":104,"area":null},{"id":"2204.06618","title":"Formal Language Recognition by Hard Attention Transformers: Perspectives from Circuit Complexity","created":"2022-04-13","cited_by_count":93,"area":null},{"id":"2402.09963","title":"Why are Sensitive Functions Hard for Transformers?","created":"2024-02-15","cited_by_count":74,"area":null},{"id":"2512.24695","title":"Nested Learning: The Illusion of Deep Learning Architectures","created":"2025-12-31","cited_by_count":72,"area":null},{"id":"2211.12316","title":"Simplicity Bias in Transformers and their Ability to Learn Sparse Boolean Functions","created":"2022-11-22","cited_by_count":70,"area":null},{"id":"2301.10743","title":"Tighter Bounds on the Expressivity of Transformer Encoders","created":"2023-01-25","cited_by_count":66,"area":null},{"id":"2402.18510","title":"RNNs are not Transformers (Yet): The Key Bottleneck on In-context Retrieval","created":"2024-02-28","cited_by_count":64,"area":null},{"id":"2310.03016","title":"Understanding In-Context Learning in Transformers and LLMs by Learning to Learn Discrete Functions","created":"2023-10-04","cited_by_count":63,"area":null},{"id":"2305.18741","title":"Grokking of Hierarchical Structure in Vanilla Transformers","created":"2023-05-30","cited_by_count":53,"area":null},{"id":"2503.03961","title":"A Little Depth Goes a Long Way: The Expressive Power of Log-Depth Transformers","created":"2025-03-05","cited_by_count":53,"area":null},{"id":"2402.11917","title":"A Mechanistic Analysis of a Transformer Trained on a Symbolic Multi-Step Reasoning Task","created":"2024-02-19","cited_by_count":49,"area":null},{"id":"2303.08117","title":"Do Transformers Parse while Predicting the Masked Word?","created":"2023-03-14","cited_by_count":48,"area":null},{"id":"2307.14023","title":"Are Transformers with One Layer Self-Attention Using Low-Rank Weight Matrices Universal Approximators?","created":"2023-07-26","cited_by_count":47,"area":null},{"id":"2303.16504","title":"An Over-parameterized Exponential Regression","created":"2023-03-29","cited_by_count":45,"area":null},{"id":"2310.19956","title":"The Impact of Depth on Compositional Generalization in Transformer Language Models","created":"2023-10-30","cited_by_count":38,"area":null},{"id":"2503.02854","title":"(How) Do Language Models Track State?","created":"2025-03-04","cited_by_count":34,"area":null},{"id":"2404.14994","title":"Transformers Can Represent $n$-gram Language Models","created":"2024-04-23","cited_by_count":32,"area":null},{"id":"2502.12170","title":"MUDDFormer: Breaking Residual Bottlenecks in Transformers via Multiway Dynamic Dense Connections","created":"2025-02-13","cited_by_count":31,"area":null},{"id":"2308.03212","title":"Average-Hard Attention Transformers are Constant-Depth Uniform Threshold Circuits","created":"2023-08-06","cited_by_count":26,"area":null},{"id":"2411.07602","title":"Circuit Complexity Bounds for RoPE-based Transformer Architecture","created":"2024-11-12","cited_by_count":26,"area":null},{"id":"2405.20131","title":"Language Models Need Inductive Biases to Count Inductively","created":"2024-05-30","cited_by_count":25,"area":null},{"id":"2402.04084","title":"Provably learning a multi-head attention layer","created":"2024-02-06","cited_by_count":21,"area":null},{"id":"2404.07647","title":"Why do small language models underperform? Studying Language Model Saturation via the Softmax Bottleneck","created":"2024-04-11","cited_by_count":21,"area":null},{"id":"2310.16076","title":"Practical Computational Power of Linear Transformers and Their Recurrent and Self-Referential Extensions","created":"2023-10-24","cited_by_count":20,"area":null},{"id":"2402.13934","title":"Do Efficient Transformers Really Save Computation?","created":"2024-02-21","cited_by_count":20,"area":null},{"id":"2406.04289","title":"What Languages are Easy to Language-Model? A Perspective from Learning Probabilistic Regular Languages","created":"2024-06-06","cited_by_count":20,"area":null},{"id":"2410.09375","title":"Looped ReLU MLPs May Be All You Need as Practical Programmable Computers","created":"2024-10-12","cited_by_count":19,"area":null},{"id":"2310.08540","title":"Do pretrained Transformers Learn In-Context by Gradient Descent?","created":"2023-10-12","cited_by_count":18,"area":null},{"id":"2505.23623","title":"Characterizing the Expressivity of Fixed-Precision Transformer Language Models","created":"2025-05-29","cited_by_count":14,"area":null},{"id":"2310.00726","title":"Improving Length-Generalization in Transformers via Task Hinting","created":"2023-10-01","cited_by_count":13,"area":null},{"id":"2507.12549","title":"The Serial Scaling Hypothesis","created":"2025-07-16","cited_by_count":13,"area":null},{"id":"2412.18040","title":"Theoretical Constraints on the Expressive Power of $\\mathsf{RoPE}$-based Tensor Attention Transformers","created":"2024-12-23","cited_by_count":12,"area":null},{"id":"2501.04299","title":"Circuit Complexity Bounds for Visual Autoregressive Model","created":"2025-01-08","cited_by_count":10,"area":null},{"id":"2502.16490","title":"On Computational Limits of FlowAR Models: Expressivity and Efficiency","created":"2025-02-23","cited_by_count":10,"area":null},{"id":"2503.14615","title":"Unique Hard Attention: A Tale of Two Sides","created":"2025-03-18","cited_by_count":10,"area":null},{"id":"2412.05562","title":"Modern Hopfield Networks Require Chain-of-Thought to Solve $\\mathsf{NC}^1$-Hard Problems","created":"2024-12-07","cited_by_count":9,"area":null},{"id":"2404.02040","title":"Transformers as Transducers","created":"2024-04-02","cited_by_count":8,"area":null},{"id":"2410.11474","title":"How Transformers Get Rich: Approximation and Dynamics Analysis","created":"2024-10-15","cited_by_count":8,"area":null},{"id":"2603.14360","title":"M$^2$RNN: Non-Linear RNNs with Matrix-Valued States for Scalable Language Modeling","created":"2026-03-15","cited_by_count":8,"area":null},{"id":"2406.02585","title":"Contextual Counting: A Mechanistic Study of Transformers on a Quantitative Task","created":"2024-05-30","cited_by_count":7,"area":null},{"id":"2310.05161","title":"Recurrent Neural Language Models as Probabilistic Finite-state Automata","created":"2023-10-08","cited_by_count":6,"area":null},{"id":"2402.15478","title":"Transformers are Expressive, But Are They Expressive Enough for Regression?","created":"2024-02-23","cited_by_count":6,"area":null},{"id":"2510.00368","title":"The Transformer Cookbook","created":"2025-10-01","cited_by_count":6,"area":null},{"id":"2604.21215","title":"The Recurrent Transformer: Greater Effective Depth and Efficient Decoding","created":"2026-04-23","cited_by_count":6,"area":null},{"id":"2402.00976","title":"Investigating Recurrent Transformers with Dynamic Halt","created":"2024-02-01","cited_by_count":5,"area":null},{"id":"2405.18548","title":"Transformer Encoder Satisfiability: Complexity and Impact on Formal Reasoning","created":"2024-05-28","cited_by_count":5,"area":null},{"id":"2410.01600","title":"ENTP: Encoder-only Next Token Prediction","created":"2024-10-02","cited_by_count":5,"area":null},{"id":"2410.10101","title":"Learning Linear Attention in Polynomial Time","created":"2024-10-14","cited_by_count":5,"area":null},{"id":"2602.05896","title":"Parity, Sensitivity, and Transformers","created":"2026-02-05","cited_by_count":5,"area":null},{"id":"2403.09728","title":"Simulating Weighted Automata over Sequences and Trees with Transformers","created":"2024-03-12","cited_by_count":4,"area":null},{"id":"2405.19222","title":"Lower Bounds on the Expressivity of Recurrent Neural Language Models","created":"2024-05-29","cited_by_count":4,"area":null},{"id":"2505.19531","title":"Minimalist Softmax Attention Provably Learns Constrained Boolean Functions","created":"2025-05-26","cited_by_count":4,"area":null},{"id":"2601.01754","title":"Context-Free Recognition with Transformers","created":"2026-01-05","cited_by_count":4,"area":null},{"id":"2402.17805","title":"Graph Neural Networks and Arithmetic Circuits","created":"2024-02-27","cited_by_count":3,"area":null},{"id":"2509.09001","title":"Fast attention mechanisms: a tale of parallelism","created":"2025-09-10","cited_by_count":3,"area":null},{"id":"2512.09892","title":"Provably Learning from Modern Language Models via Low Logit Rank","created":"2025-12-10","cited_by_count":3,"area":null},{"id":"2602.11175","title":"Barriers to Discrete Reasoning with Transformers: A Survey Across Depth, Exactness, and Bandwidth","created":"2026-01-19","cited_by_count":3,"area":null},{"id":"2309.00857","title":"Evaluating Transformer's Ability to Learn Mildly Context-Sensitive Languages","created":"2023-09-02","cited_by_count":2,"area":null},{"id":"2407.04899","title":"Algorithmic Language Models with Neurally Compiled Libraries","created":"2024-07-06","cited_by_count":2,"area":null},{"id":"2506.12220","title":"Two Heads Are Better than One: Simulating Large Transformers with Small Ones","created":"2025-06-13","cited_by_count":2,"area":null},{"id":"2508.08293","title":"Topos Theory for Generative AI and LLMs","created":"2025-08-05","cited_by_count":2,"area":null},{"id":"2511.08028","title":"Generalizable Insights for Graph Transformers in Theory and Practice","created":"2025-11-11","cited_by_count":2,"area":null},{"id":"2605.22471","title":"Lost in Tokenization: Fundamental Trade-offs in Graph Tokenization for Transformers","created":"2026-05-21","cited_by_count":2,"area":null},{"id":"2606.09731","title":"Tight Sample Complexity of Transformers","created":"2026-06-08","cited_by_count":2,"area":null},{"id":"2312.09048","title":"On The Expressivity of Recurrent Neural Cascades","created":"2023-12-14","cited_by_count":1,"area":null},{"id":"2402.15814","title":"On Efficiently Representing Regular Languages as RNNs","created":"2024-02-24","cited_by_count":1,"area":null},{"id":"2407.10949","title":"Representing Rule-based Chatbots with Transformers","created":"2024-07-15","cited_by_count":1,"area":null},{"id":"2503.23174","title":"TRA: Better Length Generalisation with Threshold Relative Attention","created":"2025-03-29","cited_by_count":1,"area":null},{"id":"2506.18739","title":"On the Existence of Universal Simulators of Attention","created":"2025-06-23","cited_by_count":1,"area":null},{"id":"2508.10018","title":"A Rose by Any Other Name Would Smell as Sweet: Categorical Homotopy Theory for Large Language Models","created":"2025-08-07","cited_by_count":1,"area":null},{"id":"2509.19161","title":"Realizable Circuit Complexity: Embedding Computation in Space-Time","created":"2025-09-23","cited_by_count":1,"area":null},{"id":"2511.06232","title":"Scaling Laws and In-Context Learning: A Unified Theoretical Framework","created":"2025-11-09","cited_by_count":1,"area":null},{"id":"2601.08061","title":"Universal computation is intrinsic to language model decoding","created":"2026-01-12","cited_by_count":1,"area":null},{"id":"2602.07599","title":"Rational Transductors","created":"2026-02-07","cited_by_count":1,"area":null},{"id":"2603.11161","title":"Algorithmic Task Capture, Computational Complexity, and Inductive Bias of Infinite Transformers","created":"2026-03-11","cited_by_count":1,"area":null},{"id":"2604.14727","title":"Geometric Capacity of Transformers: A Tropical Geometry Perspective","created":"2026-04-16","cited_by_count":1,"area":null},{"id":"2605.30523","title":"Revisiting Padded Transformer Expressivity: Which Architectural Choices Matter and Which Don't","created":"2026-05-28","cited_by_count":1,"area":null},{"id":"2605.31558","title":"Positional versus Symbolic Attention Heads: Learning Dynamics, RoPE Geometry, and Length Generalization","created":"2026-05-29","cited_by_count":1,"area":null},{"id":"2606.29389","title":"Exploring the Cryptographic Limits of Transformer Networks","created":"2026-06-28","cited_by_count":1,"area":null},{"id":"2502.03490","title":"Examining Two Hop Reasoning Through Information Content Scaling","created":"2025-02-05","cited_by_count":0,"area":null},{"id":"2503.20088","title":"Generative Linguistics, Large Language Models, and the Social Nature of Scientific Success","created":"2025-03-25","cited_by_count":0,"area":null},{"id":"2506.03370","title":"Comparison of different Unique hard attention transformer models by the formal languages they can recognize","created":"2025-06-03","cited_by_count":0,"area":null},{"id":"2510.03784","title":"Performance-Efficiency Tradeoffs in Transformers: An Approximation Theory Perspective","created":"2025-10-04","cited_by_count":0,"area":null},{"id":"2510.05494","title":"Fundamental Limits of Crystalline Equivariant Graph Neural Networks: A Circuit Complexity Perspective","created":"2025-10-07","cited_by_count":0,"area":null},{"id":"2602.02834","title":"What Structural Inductive Bias Helps Transformers Reason Over Knowledge Graphs? A Study with Tabula RASA","created":"2026-02-02","cited_by_count":0,"area":null},{"id":"2602.03970","title":"Statistical Guarantees for Reasoning Probes on Looped Boolean Circuits","created":"2026-02-03","cited_by_count":0,"area":null},{"id":"2603.11332","title":"On the Computational Hardness of Transformers","created":"2026-03-11","cited_by_count":0,"area":null},{"id":"2603.23823","title":"Circuit Complexity of Hierarchical Knowledge Tracing and Implications for Log-Precision Transformers","created":"2026-03-25","cited_by_count":0,"area":null},{"id":"2605.16640","title":"Provably Shorter Scratchpads in Hybrid DeltaNet-Attention Decoders","created":"2026-05-15","cited_by_count":0,"area":null},{"id":"2605.22964","title":"Certification from Examples is Hard for Circuits and Transformers under Minimal Overparametrization","created":"2026-05-21","cited_by_count":0,"area":null},{"id":"2607.11760","title":"From Expressivity to Sample Complexity: Narrow Teachers for Transformers via C-RASP","created":"2026-07-13","cited_by_count":0,"area":null},{"id":"2607.26988","title":"A Compositional Theory of Causally Masked Transformers","created":"2026-07-29","cited_by_count":0,"area":null},{"id":"2607.29496","title":"Transcript-Managed Transformers: Monotone Multi-Agent Collapse and Universality with Two Pop-Enabled Transcripts","created":"2026-07-31","cited_by_count":0,"area":null}],"next_cursor":"100"}