{"total":199,"rows":[{"id":"2410.05229","title":"GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Models","created":"2024-10-07","cited_by_count":550,"area":"lr"},{"id":"2401.03065","title":"CRUXEval: A Benchmark for Code Reasoning, Understanding and Execution","created":"2024-01-05","cited_by_count":300,"area":null},{"id":"2402.01032","title":"Repeat After Me: Transformers are Better than State Space Models at Copying","created":"2024-02-01","cited_by_count":216,"area":null},{"id":"2311.00208","title":"What Formal Languages Can Transformers Express? A Survey","created":"2023-11-01","cited_by_count":124,"area":null},{"id":"2311.12786","title":"Mechanistically analyzing the effects of fine-tuning on procedurally defined tasks","created":"2023-11-21","cited_by_count":115,"area":null},{"id":"2402.14903","title":"Tokenization counts: the impact of tokenization on arithmetic in frontier LLMs","created":"2024-02-22","cited_by_count":115,"area":null},{"id":"2402.01761","title":"Rethinking Interpretability in the Era of Large Language Models","created":"2024-01-30","cited_by_count":111,"area":"lr"},{"id":"2407.20311","title":"Physics of Language Models: Part 2.1, Grade-School Math and the Hidden Reasoning Process","created":"2024-07-29","cited_by_count":108,"area":"lr"},{"id":"2402.04248","title":"Can Mamba Learn How to Learn? A Comparative Study on In-Context Learning Tasks","created":"2024-02-06","cited_by_count":105,"area":"ssm"},{"id":"2402.09371","title":"Transformers Can Achieve Length Generalization But Not Robustly","created":"2024-02-14","cited_by_count":86,"area":null},{"id":"2405.00208","title":"A Primer on the Inner Workings of Transformer-based Language Models","created":"2024-04-30","cited_by_count":85,"area":null},{"id":"2405.17399","title":"Transformers Can Do Arithmetic with the Right Embeddings","created":"2024-05-27","cited_by_count":78,"area":null},{"id":"2402.09963","title":"Why are Sensitive Functions Hard for Transformers?","created":"2024-02-15","cited_by_count":74,"area":null},{"id":"2402.08939","title":"Premise Order Matters in Reasoning with Large Language Models","created":"2024-02-14","cited_by_count":72,"area":"lr"},{"id":"2503.01067","title":"All Roads Lead to Likelihood: The Value of Reinforcement Learning in Fine-Tuning","created":"2025-03-03","cited_by_count":59,"area":"alignment"},{"id":"2502.14546","title":"Position: Graph Learning Will Lose Relevance Due To Poor Benchmarks","created":"2025-02-20","cited_by_count":53,"area":null},{"id":"2408.16293","title":"Physics of Language Models: Part 2.2, How to Learn From Mistakes on Grade-School Math Problems","created":"2024-08-29","cited_by_count":50,"area":"lr"},{"id":"2405.19592","title":"Why Larger Language Models Do In-context Learning Differently?","created":"2024-05-30","cited_by_count":45,"area":null},{"id":"2403.02241","title":"Neural Redshift: Random Networks are not Random Functions","created":"2024-03-04","cited_by_count":43,"area":null},{"id":"2411.03766","title":"Number Cookbook: Number Understanding of Language Models and How to Improve It","created":"2024-11-06","cited_by_count":42,"area":null},{"id":"2406.09519","title":"Talking Heads: Understanding Inter-layer Communication in Transformer Language Models","created":"2024-06-13","cited_by_count":41,"area":null},{"id":"2401.06751","title":"The Unreasonable Effectiveness of Easy Training Data for Hard Tasks","created":"2024-01-12","cited_by_count":40,"area":"lr"},{"id":"2404.04393","title":"Counting Like Transformers: Compiling Temporal Counting Logic Into Softmax Transformers","created":"2024-04-05","cited_by_count":39,"area":null},{"id":"2502.19249","title":"Between Circuits and Chomsky: Pre-pretraining on Formal Languages Imparts Linguistic Biases","created":"2025-02-26","cited_by_count":37,"area":null},{"id":"2406.06467","title":"How Far Can Transformers Reason? The Globality Barrier and Inductive Scratchpad","created":"2024-06-10","cited_by_count":36,"area":null},{"id":"2312.17044","title":"Length Extrapolation of Transformers: A Survey from the Perspective of Positional Encoding","created":"2023-12-28","cited_by_count":32,"area":null},{"id":"2405.14722","title":"DAPE: Data-Adaptive Positional Encoding for Length Extrapolation","created":"2024-05-23","cited_by_count":31,"area":null},{"id":"2502.01612","title":"Self-Improving Transformers Overcome Easy-to-Hard and Length Generalization Challenges","created":"2025-02-03","cited_by_count":31,"area":"synth"},{"id":"2406.09347","title":"Separations in the Representational Capabilities of Transformers and Recurrent Architectures","created":"2024-06-13","cited_by_count":29,"area":null},{"id":"2501.05414","title":"LongProc: Benchmarking Long-Context Language Models on Long Procedural Generation","created":"2025-01-09","cited_by_count":29,"area":null},{"id":"2406.04229","title":"The CLRS-Text Algorithmic Reasoning Language Benchmark","created":"2024-06-06","cited_by_count":28,"area":"lr"},{"id":"2408.09503","title":"Out-of-distribution generalization via composition: a lens through induction heads in Transformers","created":"2024-08-18","cited_by_count":28,"area":null},{"id":"2502.02393","title":"Lower Bounds for Chain-of-Thought Reasoning in Hard-Attention Transformers","created":"2025-02-04","cited_by_count":28,"area":"lr"},{"id":"2410.02703","title":"Selective Attention Improves Transformer","created":"2024-10-03","cited_by_count":26,"area":null},{"id":"2405.20131","title":"Language Models Need Inductive Biases to Count Inductively","created":"2024-05-30","cited_by_count":25,"area":null},{"id":"2407.03310","title":"Universal Length Generalization with Turing Programs","created":"2024-07-03","cited_by_count":25,"area":null},{"id":"2405.20671","title":"Position Coupling: Improving Length Generalization of Arithmetic Transformers Using Task Structure","created":"2024-05-31","cited_by_count":23,"area":null},{"id":"2311.12997","title":"Compositional Capabilities of Autoregressive Transformers: A Study on Synthetic, Interpretable Tasks","created":"2023-11-21","cited_by_count":22,"area":null},{"id":"2402.03268","title":"Understanding Reasoning Ability of Language Models From the Perspective of Reasoning Paths Aggregation","created":"2024-02-05","cited_by_count":22,"area":"lr"},{"id":"2402.04875","title":"On Provable Length and Compositional Generalization","created":"2024-02-07","cited_by_count":22,"area":null},{"id":"2406.06893","title":"Transformers Provably Learn Sparse Token Selection While Fully-Connected Nets Cannot","created":"2024-06-11","cited_by_count":22,"area":null},{"id":"2410.11268","title":"Bypassing the Exponential Dependency: Looped Transformers Efficiently Learn In-context by Multi-step Gradient Descent","created":"2024-10-15","cited_by_count":22,"area":null},{"id":"2502.07827","title":"Implicit Language Models are RNNs: Balancing Parallelization and Expressivity","created":"2025-02-10","cited_by_count":22,"area":"lr"},{"id":"2502.09741","title":"FoNE: Precise Single-Token Number Embeddings via Fourier Features","created":"2025-02-13","cited_by_count":22,"area":null},{"id":"2410.19034","title":"Mixture of Parrots: Experts improve memorization more than reasoning","created":"2024-10-24","cited_by_count":21,"area":"moe"},{"id":"2506.16640","title":"Long-Context Generalization with Sparse Attention","created":"2025-06-19","cited_by_count":21,"area":null},{"id":"2404.01601","title":"What Can Transformer Learn with Varying Depth? Case Studies on Sequence Learning Tasks","created":"2024-04-02","cited_by_count":19,"area":null},{"id":"2410.17194","title":"Representation Shattering in Transformers: A Synthetic Study with Knowledge Editing","created":"2024-10-22","cited_by_count":19,"area":null},{"id":"2405.03207","title":"A Philosophical Introduction to Language Models - Part II: The Way Forward","created":"2024-05-06","cited_by_count":18,"area":null},{"id":"2502.20129","title":"Finite State Automata Inside Transformers with Chain-of-Thought: A Mechanistic Study on State Tracking","created":"2025-02-27","cited_by_count":18,"area":"lr"},{"id":"2411.12892","title":"Selective Attention: Enhancing Transformer through Principled Context Control","created":"2024-11-19","cited_by_count":16,"area":null},{"id":"2401.09074","title":"Code Simulation Challenges for Large Language Models","created":"2024-01-17","cited_by_count":15,"area":null},{"id":"2502.19981","title":"The Lookahead Limitation: Why Multi-Operand Addition is Hard for LLMs","created":"2025-02-27","cited_by_count":15,"area":null},{"id":"2402.02212","title":"A Survey to Recent Progress Towards Understanding In-Context Learning","created":"2024-02-03","cited_by_count":14,"area":null},{"id":"2408.04532","title":"How Transformers Utilize Multi-Head Attention in In-Context Learning? A Case Study on Sparse Linear Regression","created":"2024-08-08","cited_by_count":14,"area":null},{"id":"2511.07378","title":"Transformers Provably Learn Chain-of-Thought Reasoning with Length Generalization","created":"2025-11-10","cited_by_count":14,"area":"lr"},{"id":"2405.19162","title":"Does learning the right latent variables necessarily improve in-context learning?","created":"2024-05-29","cited_by_count":13,"area":null},{"id":"2406.09308","title":"Transformers meet Neural Algorithmic Reasoners","created":"2024-06-13","cited_by_count":13,"area":"lr"},{"id":"2503.10065","title":"Do We Always Need the Simplicity Bias? Looking for Optimal Inductive Biases in the Wild","created":"2025-03-13","cited_by_count":13,"area":null},{"id":"2506.09251","title":"Extrapolation by Association: Length Generalization Transfer in Transformers","created":"2025-06-10","cited_by_count":13,"area":null},{"id":"2507.12549","title":"The Serial Scaling Hypothesis","created":"2025-07-16","cited_by_count":13,"area":null},{"id":"2404.00560","title":"A Theory for Length Generalization in Learning to Reason","created":"2024-03-31","cited_by_count":11,"area":null},{"id":"2406.01895","title":"Explicitly Encoding Structural Symmetry is Key to Length Generalization in Arithmetic Tasks","created":"2024-06-04","cited_by_count":11,"area":"lr"},{"id":"2406.17295","title":"Less can be more for predicting properties with large language models","created":"2024-06-25","cited_by_count":11,"area":null},{"id":"2408.05506","title":"Your Context Is Not an Array: Unveiling Random Access Limitations in Transformers","created":"2024-08-10","cited_by_count":11,"area":null},{"id":"2503.19173","title":"Graph neural networks extrapolate out-of-distribution for shortest paths","created":"2025-03-24","cited_by_count":11,"area":null},{"id":"2406.04089","title":"On Limitation of Transformer for Learning HMMs","created":"2024-06-06","cited_by_count":10,"area":null},{"id":"2410.04798","title":"DAPE V2: Process Attention Score as Feature Map for Length Extrapolation","created":"2024-10-07","cited_by_count":10,"area":null},{"id":"2402.11628","title":"Discrete Neural Algorithmic Reasoning","created":"2024-02-18","cited_by_count":9,"area":null},{"id":"2505.11199","title":"The Counting Power of Transformers","created":"2025-05-16","cited_by_count":9,"area":null},{"id":"2404.02040","title":"Transformers as Transducers","created":"2024-04-02","cited_by_count":8,"area":null},{"id":"2406.11243","title":"FamiCom: Further Demystifying Prompts for Language Models with Task-Agnostic Performance Estimation","created":"2024-06-17","cited_by_count":8,"area":null},{"id":"2409.17391","title":"Scaling Behavior for Large Language Models regarding Numeral Systems: An Example using Pythia","created":"2024-09-25","cited_by_count":8,"area":null},{"id":"2410.13779","title":"The Mystery of the Pathological Path-star Task for Language Models","created":"2024-10-17","cited_by_count":8,"area":null},{"id":"2410.14086","title":"In-context learning and Occam's razor","created":"2024-10-17","cited_by_count":8,"area":null},{"id":"2411.02322","title":"LayerDAG: A Layerwise Autoregressive Diffusion Model for Directed Acyclic Graph Generation","created":"2024-11-04","cited_by_count":8,"area":"diffusion"},{"id":"2502.16792","title":"The Role of Sparsity for Length Generalization in Transformers","created":"2025-02-24","cited_by_count":8,"area":null},{"id":"2503.16048","title":"Meta-Learning Neural Mechanisms rather than Bayesian Priors","created":"2025-03-20","cited_by_count":8,"area":null},{"id":"2510.27015","title":"Quantitative Bounds for Length Generalization in Transformers","created":"2025-10-30","cited_by_count":8,"area":null},{"id":"2405.15164","title":"From Frege to chatGPT: Compositionality in language, cognition, and deep neural networks","created":"2024-05-24","cited_by_count":7,"area":null},{"id":"2408.14906","title":"Writing in the Margins: Better Inference Pattern for Long Context Retrieval","created":"2024-08-27","cited_by_count":7,"area":null},{"id":"2410.18077","title":"ALTA: Compiler-Based Analysis of Transformers","created":"2024-10-23","cited_by_count":7,"area":null},{"id":"2412.07752","title":"FlashRNN: I/O-Aware Optimization of Traditional RNNs on modern hardware","created":"2024-12-10","cited_by_count":7,"area":null},{"id":"2502.03568","title":"Code Simulation as a Proxy for High-order Tasks in Large Language Models","created":"2025-02-05","cited_by_count":7,"area":null},{"id":"2502.11525","title":"Beyond Single-Task: Robust Multi-Task Length Generalization for LLMs","created":"2025-02-17","cited_by_count":7,"area":null},{"id":"2502.17356","title":"Random Scaling of Emergent Capabilities","created":"2025-02-24","cited_by_count":7,"area":null},{"id":"2402.03774","title":"Learning a Decision Tree Algorithm with Transformers","created":"2024-02-06","cited_by_count":6,"area":null},{"id":"2504.02827","title":"On Vanishing Variance in Transformer Length Generalization","created":"2025-04-03","cited_by_count":6,"area":null},{"id":"2505.21785","title":"Born a Transformer -- Always a Transformer? On the Effect of Pretraining on Architectural Abilities","created":"2025-05-27","cited_by_count":6,"area":null},{"id":"2509.22445","title":"Bridging Kolmogorov Complexity and Deep Learning: Asymptotically Optimal Description Length Objectives for Transformers","created":"2025-09-26","cited_by_count":6,"area":null},{"id":"2512.18634","title":"From Shortcut to Induction Head: How Data Diversity Shapes Algorithm Selection in Transformers","created":"2025-12-21","cited_by_count":6,"area":null},{"id":"2402.00976","title":"Investigating Recurrent Transformers with Dynamic Halt","created":"2024-02-01","cited_by_count":5,"area":null},{"id":"2410.01600","title":"ENTP: Encoder-only Next Token Prediction","created":"2024-10-02","cited_by_count":5,"area":null},{"id":"2410.08309","title":"Swing-by Dynamics in Concept Learning and Compositional Generalization","created":"2024-10-10","cited_by_count":5,"area":"diffusion"},{"id":"2411.05943","title":"Quantifying artificial intelligence through algorithmic generalization","created":"2024-11-08","cited_by_count":5,"area":null},{"id":"2502.10450","title":"Trustworthy AI: Safety, Bias, and Privacy -- A Survey","created":"2025-02-11","cited_by_count":5,"area":"safety"},{"id":"2507.07544","title":"Position: We Need An Algorithmic Understanding of Generative AI","created":"2025-07-10","cited_by_count":5,"area":null},{"id":"2402.07875","title":"Implicit Bias of Policy Gradient in Linear Quadratic Control: Extrapolation to Unseen Initial States","created":"2024-02-12","cited_by_count":4,"area":"deeprl"},{"id":"2407.14055","title":"Quantum Hamiltonian Embedding of Images for Data Reuploading Classifiers","created":"2024-07-19","cited_by_count":4,"area":null},{"id":"2502.15609","title":"On the Robustness of Transformers against Context Hijacking for Linear Classification","created":"2025-02-21","cited_by_count":4,"area":null}],"next_cursor":"100"}