{"total":268,"rows":[{"id":"2303.12712","title":"Sparks of Artificial General Intelligence: Early experiments with GPT-4","created":"2023-03-22","cited_by_count":2592,"area":null},{"id":"2307.05782","title":"Large Language Models","created":"2023-07-11","cited_by_count":780,"area":null},{"id":"2305.18654","title":"Faith and Fate: Limits of Transformers on Compositionality","created":"2023-05-29","cited_by_count":503,"area":null},{"id":"2306.09927","title":"Trained Transformers Learn Linear Models In-Context","created":"2023-06-16","cited_by_count":337,"area":null},{"id":"2306.04637","title":"Transformers as Statisticians: Provable In-Context Learning with In-Context Algorithm Selection","created":"2023-06-07","cited_by_count":298,"area":null},{"id":"2305.15408","title":"Towards Revealing the Mystery behind Chain of Thought: A Theoretical Perspective","created":"2023-05-24","cited_by_count":294,"area":"lr"},{"id":"2409.12183","title":"To CoT or not to CoT? Chain-of-thought helps mainly on math and symbolic reasoning","created":"2024-09-18","cited_by_count":263,"area":"lr"},{"id":"2402.12875","title":"Chain of Thought Empowers Transformers to Solve Inherently Serial Problems","created":"2024-02-20","cited_by_count":260,"area":"lr"},{"id":"2502.17416","title":"Reasoning with Latent Thoughts: On the Power of Looped Transformers","created":"2025-02-24","cited_by_count":240,"area":"lr"},{"id":"2301.13196","title":"Looped Transformers as Programmable Computers","created":"2023-01-30","cited_by_count":237,"area":null},{"id":"2404.08819","title":"The Illusion of State in State-Space Models","created":"2024-04-12","cited_by_count":187,"area":null},{"id":"2307.03576","title":"One Step of Gradient Descent is Provably the Optimal In-Context Learner with One Layer of Linear Self-Attention","created":"2023-07-07","cited_by_count":170,"area":null},{"id":"2510.25741","title":"Scaling Latent Reasoning via Looped Language Models","created":"2025-10-29","cited_by_count":165,"area":"lr"},{"id":"2306.00802","title":"Birth of a Transformer: A Memory Viewpoint","created":"2023-06-01","cited_by_count":161,"area":null},{"id":"2403.06963","title":"The pitfalls of next-token prediction","created":"2024-03-11","cited_by_count":160,"area":null},{"id":"2311.12424","title":"Looped Transformers are Better at Learning Learning Algorithms","created":"2023-11-21","cited_by_count":149,"area":null},{"id":"2306.02896","title":"Representational Strengths and Limitations of Transformers","created":"2023-06-05","cited_by_count":133,"area":null},{"id":"2311.00208","title":"What Formal Languages Can Transformers Express? A Survey","created":"2023-11-01","cited_by_count":124,"area":null},{"id":"2311.12786","title":"Mechanistically analyzing the effects of fine-tuning on procedurally defined tasks","created":"2023-11-21","cited_by_count":115,"area":null},{"id":"2406.03689","title":"Evaluating the World Model Implicit in a Generative Model","created":"2024-06-06","cited_by_count":113,"area":"wm"},{"id":"2402.04248","title":"Can Mamba Learn How to Learn? A Comparative Study on In-Context Learning Tasks","created":"2024-02-06","cited_by_count":105,"area":"ssm"},{"id":"2411.12537","title":"Unlocking State-Tracking in Linear RNNs Through Negative Eigenvalues","created":"2024-11-19","cited_by_count":101,"area":"ssm"},{"id":"2309.05858","title":"Uncovering mesa-optimization algorithms in Transformers","created":"2023-09-11","cited_by_count":100,"area":null},{"id":"2303.04245","title":"How Do Transformers Learn Topic Structure: Towards a Mechanistic Understanding","created":"2023-03-07","cited_by_count":86,"area":null},{"id":"2402.09371","title":"Transformers Can Achieve Length Generalization But Not Robustly","created":"2024-02-14","cited_by_count":86,"area":null},{"id":"2310.08566","title":"Transformers as Decision Makers: Provable In-Context Reinforcement Learning via Supervised Pretraining","created":"2023-10-12","cited_by_count":83,"area":"offlinerl"},{"id":"2402.09268","title":"Transformers, parallel computation, and logarithmic depth","created":"2024-02-14","cited_by_count":79,"area":null},{"id":"2405.15071","title":"Grokked Transformers are Implicit Reasoners: A Mechanistic Journey to the Edge of Generalization","created":"2024-05-23","cited_by_count":79,"area":"lr"},{"id":"2405.15943","title":"Transformers represent belief state geometry in their residual stream","created":"2024-05-24","cited_by_count":77,"area":null},{"id":"2505.12514","title":"Reasoning by Superposition: A Theoretical Perspective on Chain of Continuous Thought","created":"2025-05-18","cited_by_count":77,"area":"lr"},{"id":"2306.00946","title":"Exposing Attention Glitches with Flip-Flop Language Modeling","created":"2023-06-01","cited_by_count":75,"area":null},{"id":"2502.10297","title":"DeltaProduct: Improving State-Tracking in Linear RNNs via Householder Products","created":"2025-02-14","cited_by_count":75,"area":null},{"id":"2309.06979","title":"Auto-Regressive Next-Token Predictors are Universal Learners","created":"2023-09-13","cited_by_count":67,"area":"lr"},{"id":"2212.13894","title":"LAMBADA: Backward Chaining for Automated Reasoning in Natural Language","created":"2022-12-20","cited_by_count":66,"area":"lr"},{"id":"2402.18510","title":"RNNs are not Transformers (Yet): The Key Bottleneck on In-context Retrieval","created":"2024-02-28","cited_by_count":64,"area":null},{"id":"2304.05366","title":"The No Free Lunch Theorem, Kolmogorov Complexity, and the Role of Inductive Biases in Machine Learning","created":"2023-04-11","cited_by_count":63,"area":null},{"id":"2310.03016","title":"Understanding In-Context Learning in Transformers and LLMs by Learning to Learn Discrete Functions","created":"2023-10-04","cited_by_count":63,"area":null},{"id":"2411.01855","title":"Can Language Models Learn to Skip Steps?","created":"2024-11-04","cited_by_count":59,"area":"lr"},{"id":"2402.18312","title":"How to think step-by-step: A mechanistic understanding of chain-of-thought reasoning","created":"2024-02-28","cited_by_count":53,"area":"lr"},{"id":"2503.03961","title":"A Little Depth Goes a Long Way: The Expressive Power of Log-Depth Transformers","created":"2025-03-05","cited_by_count":53,"area":null},{"id":"2310.07923","title":"The Expressive Power of Transformers with Chain of Thought","created":"2023-10-11","cited_by_count":50,"area":"lr"},{"id":"2501.00070","title":"ICLR: In-Context Learning of Representations","created":"2024-12-29","cited_by_count":50,"area":null},{"id":"2303.08117","title":"Do Transformers Parse while Predicting the Masked Word?","created":"2023-03-14","cited_by_count":48,"area":null},{"id":"2507.06952","title":"What Has a Foundation Model Found? Using Inductive Bias to Probe for World Models","created":"2025-07-09","cited_by_count":47,"area":"wm"},{"id":"2405.17394","title":"The Expressive Capacity of State Space Models: A Formal Language Perspective","created":"2024-05-27","cited_by_count":46,"area":null},{"id":"2303.16504","title":"An Over-parameterized Exponential Regression","created":"2023-03-29","cited_by_count":45,"area":null},{"id":"2412.02975","title":"Theoretical limitations of multi-layer Transformer","created":"2024-12-04","cited_by_count":45,"area":null},{"id":"2307.11353","title":"What can a Single Attention Layer Learn? A Study Through the Random Features Lens","created":"2023-07-21","cited_by_count":40,"area":null},{"id":"2401.15530","title":"An Information-Theoretic Analysis of In-Context Learning","created":"2024-01-28","cited_by_count":40,"area":null},{"id":"2309.11420","title":"Deep Networks as Denoising Algorithms: Sample-Efficient Learning of Diffusion Models in High-Dimensional Graphical Models","created":"2023-09-20","cited_by_count":39,"area":"diffusion"},{"id":"2409.10502","title":"Causal Language Modeling Can Elicit Search and Reasoning Capabilities on Logic Puzzles","created":"2024-09-16","cited_by_count":38,"area":"lr"},{"id":"2503.02854","title":"(How) Do Language Models Track State?","created":"2025-03-04","cited_by_count":34,"area":null},{"id":"2505.16381","title":"PaTH Attention: Position Encoding via Accumulating Householder Transformations","created":"2025-05-22","cited_by_count":34,"area":null},{"id":"2310.07096","title":"Sparse Universal Transformer","created":"2023-10-11","cited_by_count":32,"area":"moe"},{"id":"2502.21212","title":"Transformers Learn to Implement Multi-step Gradient Descent with Chain of Thought","created":"2025-02-28","cited_by_count":32,"area":"lr"},{"id":"2401.14953","title":"Learning Universal Predictors","created":"2024-01-26","cited_by_count":31,"area":null},{"id":"2511.05963","title":"Next-Latent Prediction Transformers Learn Compact World Models","created":"2025-11-08","cited_by_count":31,"area":"wm"},{"id":"2312.01429","title":"Transformers are uninterpretable with myopic methods: a case study with bounded Dyck grammars","created":"2023-12-03","cited_by_count":30,"area":null},{"id":"2402.01107","title":"Simulation of Graph Algorithms with Looped Transformers","created":"2024-02-02","cited_by_count":30,"area":null},{"id":"2405.16411","title":"Training Tensor Attention Efficiently: From Cubic to Almost Linear Time","created":"2024-05-26","cited_by_count":30,"area":null},{"id":"2406.09347","title":"Separations in the Representational Capabilities of Transformers and Recurrent Architectures","created":"2024-06-13","cited_by_count":29,"area":null},{"id":"2504.15266","title":"Roll the dice & look before you leap: Going beyond the creative limits of next-token prediction","created":"2025-04-21","cited_by_count":29,"area":null},{"id":"2405.04669","title":"Towards a Theoretical Understanding of the 'Reversal Curse' via Training Dynamics","created":"2024-05-07","cited_by_count":27,"area":"lr"},{"id":"2305.15287","title":"The Crucial Role of Normalization in Sharpness-Aware Minimization","created":"2023-05-24","cited_by_count":26,"area":null},{"id":"2308.03212","title":"Average-Hard Attention Transformers are Constant-Depth Uniform Threshold Circuits","created":"2023-08-06","cited_by_count":26,"area":null},{"id":"2408.12578","title":"A Percolation Model of Emergence: Analyzing Transformers Trained on a Formal Language","created":"2024-08-22","cited_by_count":26,"area":null},{"id":"2411.07602","title":"Circuit Complexity Bounds for RoPE-based Transformer Architecture","created":"2024-11-12","cited_by_count":26,"area":null},{"id":"2305.16704","title":"A Closer Look at In-Context Learning under Distribution Shifts","created":"2023-05-26","cited_by_count":25,"area":null},{"id":"2312.03656","title":"Interpretability Illusions in the Generalization of Simplified Models","created":"2023-12-06","cited_by_count":25,"area":null},{"id":"2402.00522","title":"Understanding the Expressive Power and Mechanisms of Transformer for Sequence Modeling","created":"2024-02-01","cited_by_count":25,"area":null},{"id":"2405.20131","title":"Language Models Need Inductive Biases to Count Inductively","created":"2024-05-30","cited_by_count":25,"area":null},{"id":"2407.03310","title":"Universal Length Generalization with Turing Programs","created":"2024-07-03","cited_by_count":25,"area":null},{"id":"2401.16421","title":"Two Stones Hit One Bird: Bilevel Positional Encoding for Better Length Extrapolation","created":"2024-01-29","cited_by_count":24,"area":null},{"id":"2407.10005","title":"Fine-grained Analysis of In-context Linear Estimation: Data, Architecture, and Beyond","created":"2024-07-13","cited_by_count":24,"area":null},{"id":"2408.07144","title":"Language Models as Models of Language","created":"2024-08-13","cited_by_count":24,"area":null},{"id":"2411.07107","title":"Training Neural Networks as Recognizers of Formal Languages","created":"2024-11-11","cited_by_count":24,"area":"lr"},{"id":"2311.04128","title":"Generative learning for nonlinear dynamics","created":"2023-11-07","cited_by_count":22,"area":null},{"id":"2311.12997","title":"Compositional Capabilities of Autoregressive Transformers: A Study on Synthetic, Interpretable Tasks","created":"2023-11-21","cited_by_count":22,"area":null},{"id":"2402.03268","title":"Understanding Reasoning Ability of Language Models From the Perspective of Reasoning Paths Aggregation","created":"2024-02-05","cited_by_count":22,"area":"lr"},{"id":"2402.11639","title":"In-Context Learning with Transformers: Softmax Attention Adapts to Function Lipschitzness","created":"2024-02-18","cited_by_count":22,"area":null},{"id":"2406.06893","title":"Transformers Provably Learn Sparse Token Selection While Fully-Connected Nets Cannot","created":"2024-06-11","cited_by_count":22,"area":null},{"id":"2412.06148","title":"The Computational Limits of State-Space Models and Mamba via the Lens of Circuit Complexity","created":"2024-12-09","cited_by_count":21,"area":"ssm"},{"id":"2402.13934","title":"Do Efficient Transformers Really Save Computation?","created":"2024-02-21","cited_by_count":20,"area":null},{"id":"2305.18869","title":"Dissecting Chain-of-Thought: Compositionality through In-Context Filtering and Learning","created":"2023-05-30","cited_by_count":19,"area":"lr"},{"id":"2404.01601","title":"What Can Transformer Learn with Varying Depth? Case Studies on Sequence Learning Tasks","created":"2024-04-02","cited_by_count":19,"area":null},{"id":"2405.16674","title":"Limits of Deep Learning: Sequence Modeling through the Lens of Complexity Theory","created":"2024-05-26","cited_by_count":19,"area":null},{"id":"2410.09375","title":"Looped ReLU MLPs May Be All You Need as Practical Programmable Computers","created":"2024-10-12","cited_by_count":19,"area":null},{"id":"2310.08540","title":"Do pretrained Transformers Learn In-Context by Gradient Descent?","created":"2023-10-12","cited_by_count":18,"area":null},{"id":"2502.20129","title":"Finite State Automata Inside Transformers with Chain-of-Thought: A Mechanistic Study on State Tracking","created":"2025-02-27","cited_by_count":18,"area":"lr"},{"id":"2505.17761","title":"Structured Linear CDEs: Maximally Expressive and Parallel-in-Time Sequence Models","created":"2025-05-23","cited_by_count":18,"area":null},{"id":"2510.04401","title":"Your Vision-Language Model Can't Even Count to 20: Exposing the Failures of VLMs in Compositional Counting","created":"2025-10-06","cited_by_count":18,"area":"mllm"},{"id":"2305.14699","title":"Can Transformers Learn to Solve Problems Recursively?","created":"2023-05-24","cited_by_count":17,"area":null},{"id":"2307.01189","title":"Trainable Transformer in Transformer","created":"2023-07-03","cited_by_count":17,"area":null},{"id":"2404.18444","title":"U-Nets as Belief Propagation: Efficient Classification, Denoising, and Diffusion in Generative Hierarchical Models","created":"2024-04-29","cited_by_count":17,"area":"diffusion"},{"id":"2405.05409","title":"Initialization is Critical to Whether Transformers Fit Composite Functions by Reasoning or Memorizing","created":"2024-05-08","cited_by_count":17,"area":null},{"id":"2410.07432","title":"Can Transformers Reason Logically? A Study in SAT Solving","created":"2024-10-09","cited_by_count":16,"area":"lr"},{"id":"2509.22284","title":"Structured Sparse Transition Matrices to Enable State Tracking in State-Space Models","created":"2025-09-26","cited_by_count":16,"area":null},{"id":"2509.23365","title":"Emergence of Superposition: Unveiling the Training Dynamics of Chain of Continuous Thought","created":"2025-09-27","cited_by_count":16,"area":"lr"},{"id":"2410.05464","title":"Progressive distillation induces an implicit curriculum","created":"2024-10-07","cited_by_count":15,"area":null},{"id":"2412.07942","title":"Neural Scaling Laws Rooted in the Data Distribution","created":"2024-12-10","cited_by_count":15,"area":null}],"next_cursor":"100"}