{"total":111,"rows":[{"id":"2402.01032","title":"Repeat After Me: Transformers are Better than State Space Models at Copying","created":"2024-02-01","cited_by_count":216,"area":null},{"id":"2310.16028","title":"What Algorithms can Transformers Learn? A Study in Length Generalization","created":"2023-10-24","cited_by_count":199,"area":null},{"id":"2311.00208","title":"What Formal Languages Can Transformers Express? A Survey","created":"2023-11-01","cited_by_count":124,"area":null},{"id":"2402.09268","title":"Transformers, parallel computation, and logarithmic depth","created":"2024-02-14","cited_by_count":79,"area":null},{"id":"2402.09963","title":"Why are Sensitive Functions Hard for Transformers?","created":"2024-02-15","cited_by_count":74,"area":null},{"id":"2402.08871","title":"Position: Topological Deep Learning is the New Frontier for Relational Learning","created":"2024-02-14","cited_by_count":66,"area":null},{"id":"2402.18510","title":"RNNs are not Transformers (Yet): The Key Bottleneck on In-context Retrieval","created":"2024-02-28","cited_by_count":64,"area":null},{"id":"2402.08164","title":"On Limitations of the Transformer Architecture","created":"2024-02-13","cited_by_count":63,"area":null},{"id":"2405.17394","title":"The Expressive Capacity of State Space Models: A Formal Language Perspective","created":"2024-05-27","cited_by_count":46,"area":null},{"id":"2412.02975","title":"Theoretical limitations of multi-layer Transformer","created":"2024-12-04","cited_by_count":45,"area":null},{"id":"2310.04064","title":"How to Capture Higher-order Correlations? Generalizing Matrix Softmax Attention to Kronecker Computation","created":"2023-10-06","cited_by_count":44,"area":null},{"id":"2401.15530","title":"An Information-Theoretic Analysis of In-Context Learning","created":"2024-01-28","cited_by_count":40,"area":null},{"id":"2310.12680","title":"On the Optimization and Generalization of Multi-head Attention","created":"2023-10-19","cited_by_count":37,"area":null},{"id":"2408.14332","title":"One-layer transformers fail to solve the induction heads task","created":"2024-08-26","cited_by_count":32,"area":null},{"id":"2402.05738","title":"Implicit Bias and Fast Convergence Rates for Self-attention","created":"2024-02-08","cited_by_count":31,"area":null},{"id":"2306.04933","title":"InfoPrompt: Information-Theoretic Soft Prompt Tuning for Natural Language Understanding","created":"2023-06-08","cited_by_count":30,"area":null},{"id":"2309.07418","title":"A Fast Optimization View: Reformulating Single Layer Attention in LLM Based on Tensor and SVM Trick, and Solving It in Matrix Multiplication Time","created":"2023-09-14","cited_by_count":30,"area":null},{"id":"2405.16411","title":"Training Tensor Attention Efficiently: From Cubic to Almost Linear Time","created":"2024-05-26","cited_by_count":30,"area":null},{"id":"2406.09347","title":"Separations in the Representational Capabilities of Transformers and Recurrent Architectures","created":"2024-06-13","cited_by_count":29,"area":null},{"id":"2407.17686","title":"Transformers on Markov Data: Constant Depth Suffices","created":"2024-07-25","cited_by_count":27,"area":null},{"id":"2308.12247","title":"How to Protect Copyright Data in Optimization of Large Language Models?","created":"2023-08-23","cited_by_count":26,"area":null},{"id":"2402.14753","title":"Prompting a Pretrained Transformer Can Be a Universal Approximator","created":"2024-02-22","cited_by_count":25,"area":null},{"id":"2406.13261","title":"BeHonest: Benchmarking Honesty in Large Language Models","created":"2024-06-19","cited_by_count":24,"area":null},{"id":"2407.15160","title":"When Can Transformers Count to n?","created":"2024-07-21","cited_by_count":24,"area":null},{"id":"2402.11639","title":"In-Context Learning with Transformers: Softmax Attention Adapts to Function Lipschitzness","created":"2024-02-18","cited_by_count":22,"area":null},{"id":"2406.06893","title":"Transformers Provably Learn Sparse Token Selection While Fully-Connected Nets Cannot","created":"2024-06-11","cited_by_count":22,"area":null},{"id":"2402.01929","title":"Sample, estimate, aggregate: A recipe for causal discovery foundation models","created":"2024-02-02","cited_by_count":21,"area":null},{"id":"2402.04084","title":"Provably learning a multi-head attention layer","created":"2024-02-06","cited_by_count":21,"area":null},{"id":"2504.14370","title":"Density Measures for Language Generation","created":"2025-04-19","cited_by_count":21,"area":null},{"id":"2402.13934","title":"Do Efficient Transformers Really Save Computation?","created":"2024-02-21","cited_by_count":20,"area":null},{"id":"2310.11685","title":"Why Softmax Attention Outperforms Linear Attention","created":"2023-10-18","cited_by_count":15,"area":null},{"id":"2401.15127","title":"Evaluation of LLM Chatbots for OSINT-based Cyber Threat Awareness","created":"2024-01-26","cited_by_count":14,"area":null},{"id":"2511.05295","title":"Language Generation and Identification From Partial Enumeration: Tight Density Bounds and Topological Characterizations","created":"2025-11-07","cited_by_count":13,"area":null},{"id":"2310.12462","title":"Unmasking Transformers: A Theoretical Approach to Data Recovery via Attention Weights","created":"2023-10-19","cited_by_count":12,"area":null},{"id":"2412.18040","title":"Theoretical Constraints on the Expressive Power of $\\mathsf{RoPE}$-based Tensor Attention Transformers","created":"2024-12-23","cited_by_count":12,"area":null},{"id":"2501.19215","title":"Strassen Attention, Split VC Dimension and Compositionality in Transformers","created":"2025-01-31","cited_by_count":12,"area":null},{"id":"2405.13718","title":"Next-token prediction capacity: general upper bounds and a lower bound for transformers","created":"2024-05-22","cited_by_count":11,"area":null},{"id":"2503.01805","title":"Depth-Width tradeoffs in Algorithmic Reasoning of Graph Tasks with Transformers","created":"2025-03-03","cited_by_count":11,"area":null},{"id":"2409.17677","title":"On the Optimal Memorization Capacity of Transformers","created":"2024-09-26","cited_by_count":10,"area":null},{"id":"2602.03655","title":"Sequential Group Composition: A Window into the Mechanics of Deep Learning","created":"2026-02-03","cited_by_count":10,"area":null},{"id":"2311.15390","title":"Local Convergence of Approximate Newton Method for Two Layer Nonlinear Regression","created":"2023-11-26","cited_by_count":9,"area":null},{"id":"2410.09605","title":"Training Dynamics of Transformers to Recognize Word Co-occurrence via Gradient Flow Analysis","created":"2024-10-12","cited_by_count":9,"area":null},{"id":"2506.02651","title":"Asymptotics of SGD in Sequence-Single Index Models and Single-Layer Attention Networks","created":"2025-06-03","cited_by_count":9,"area":null},{"id":"2404.02040","title":"Transformers as Transducers","created":"2024-04-02","cited_by_count":8,"area":null},{"id":"2404.03380","title":"On the Theoretical Expressive Power and the Design Space of Higher-Order Graph Transformers","created":"2024-04-04","cited_by_count":8,"area":null},{"id":"2502.16792","title":"The Role of Sparsity for Length Generalization in Transformers","created":"2025-02-24","cited_by_count":8,"area":null},{"id":"2505.21024","title":"Pause Tokens Strictly Increase the Expressivity of Constant-Depth Transformers","created":"2025-05-27","cited_by_count":8,"area":null},{"id":"2507.02754","title":"Fast and Simplex: 2-Simplicial Attention in Triton","created":"2025-07-03","cited_by_count":8,"area":null},{"id":"2308.08358","title":"Convergence of Two-Layer Regression with Nonlinear Units","created":"2023-08-16","cited_by_count":6,"area":null},{"id":"2311.13624","title":"A Theoretical Insight into Attack and Defense of Gradient Leakage in Transformer","created":"2023-11-22","cited_by_count":6,"area":null},{"id":"2311.14652","title":"One Pass Streaming Algorithm for Super Long Token Attention Approximation in Sublinear Space","created":"2023-11-24","cited_by_count":6,"area":null},{"id":"2408.15185","title":"Human-Centric Video Anomaly Detection Through Spatio-Temporal Pose Tokenization and Transformer","created":"2024-08-27","cited_by_count":6,"area":null},{"id":"2505.21785","title":"Born a Transformer -- Always a Transformer? On the Effect of Pretraining on Architectural Abilities","created":"2025-05-27","cited_by_count":6,"area":null},{"id":"2506.05200","title":"Transformers Meet In-Context Learning: A Universal Approximation Theory","created":"2025-06-05","cited_by_count":6,"area":null},{"id":"2510.00368","title":"The Transformer Cookbook","created":"2025-10-01","cited_by_count":6,"area":null},{"id":"2511.11579","title":"Decoupling Positional and Symbolic Attention Behavior in Transformers","created":"2025-10-03","cited_by_count":6,"area":null},{"id":"2407.16153","title":"On the Benefits of Rank in Attention Layers","created":"2024-07-23","cited_by_count":5,"area":null},{"id":"2410.01600","title":"ENTP: Encoder-only Next Token Prediction","created":"2024-10-02","cited_by_count":5,"area":null},{"id":"2504.13558","title":"Transformers Can Overcome the Curse of Dimensionality: A Theoretical Study from an Approximation Perspective","created":"2025-04-18","cited_by_count":5,"area":null},{"id":"2505.14840","title":"Subquadratic Algorithms and Hardness for Attention with Any Temperature","created":"2025-05-20","cited_by_count":5,"area":null},{"id":"2505.20802","title":"Leaner Transformers: More Heads, Less Depth","created":"2025-05-27","cited_by_count":5,"area":null},{"id":"2507.07544","title":"Position: We Need An Algorithmic Understanding of Generative AI","created":"2025-07-10","cited_by_count":5,"area":null},{"id":"2602.04768","title":"Billion-Scale Graph Foundation Models","created":"2026-02-04","cited_by_count":5,"area":null},{"id":"2602.05896","title":"Parity, Sensitivity, and Transformers","created":"2026-02-05","cited_by_count":5,"area":null},{"id":"2412.01417","title":"Learning Elementary Cellular Automata with Transformers","created":"2024-12-02","cited_by_count":4,"area":null},{"id":"2510.05364","title":"The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures","created":"2025-10-06","cited_by_count":4,"area":null},{"id":"2510.18870","title":"Triangle Multiplication Is All You Need For Biomolecular Structure Representations","created":"2025-10-21","cited_by_count":4,"area":null},{"id":"2407.03250","title":"When big data actually are low-rank, or entrywise approximation of certain function-generated matrices","created":"2024-07-03","cited_by_count":3,"area":null},{"id":"2409.11232","title":"Fast Analysis of the OpenAI O1-Preview Model in Solving Random K-SAT Problem: Does the LLM Solve the Problem Itself or Call an External SAT Solver?","created":"2024-09-17","cited_by_count":3,"area":null},{"id":"2409.12446","title":"Neural Networks Generalize on Low Complexity Data","created":"2024-09-19","cited_by_count":3,"area":null},{"id":"2409.18028","title":"Compositional Hardness of Code in Large Language Models -- A Probabilistic Perspective","created":"2024-09-26","cited_by_count":3,"area":null},{"id":"2411.13028","title":"A Theory for Compressibility of Graph Transformers for Transductive Learning","created":"2024-11-20","cited_by_count":3,"area":null},{"id":"2412.20195","title":"Lower bounds on transformers with infinite precision","created":"2024-12-28","cited_by_count":3,"area":null},{"id":"2501.02535","title":"A completely uniform transformer for parity","created":"2025-01-05","cited_by_count":3,"area":null},{"id":"2502.07553","title":"Transformers Provably Learn Sparse XOR with Polylogarithmic Parameters","created":"2025-02-11","cited_by_count":3,"area":null},{"id":"2503.10542","title":"Language Models, Graph Searching, and Supervision Adulteration: When More Supervision is Less and How to Make More More","created":"2025-03-13","cited_by_count":3,"area":null},{"id":"2503.11272","title":"When Do Transformers Outperform Feedforward and Recurrent Networks? A Statistical Perspective","created":"2025-03-14","cited_by_count":3,"area":null},{"id":"2503.22076","title":"Concise One-Layer Transformers Can Do Function Evaluation (Sometimes)","created":"2025-03-28","cited_by_count":3,"area":null},{"id":"2509.09001","title":"Fast attention mechanisms: a tale of parallelism","created":"2025-09-10","cited_by_count":3,"area":null},{"id":"2510.08389","title":"Revisiting Hallucination Detection with Effective Rank-based Uncertainty","created":"2025-10-09","cited_by_count":3,"area":null},{"id":"2601.17958","title":"TensorLens: End-to-End Transformer Analysis via High-Order Attention Tensors","created":"2026-01-25","cited_by_count":3,"area":null},{"id":"2602.11175","title":"Barriers to Discrete Reasoning with Transformers: A Survey Across Depth, Exactness, and Bandwidth","created":"2026-01-19","cited_by_count":3,"area":null},{"id":"2604.17935","title":"How Much Cache Does Reasoning Need? Depth-Cache Tradeoffs in KV-Compressed Transformers","created":"2026-04-20","cited_by_count":3,"area":null},{"id":"2503.10571","title":"Radar: Fast Long-Context Decoding for Any Transformer","created":"2025-03-13","cited_by_count":2,"area":null},{"id":"2506.12220","title":"Two Heads Are Better than One: Simulating Large Transformers with Small Ones","created":"2025-06-13","cited_by_count":2,"area":null},{"id":"2605.22471","title":"Lost in Tokenization: Fundamental Trade-offs in Graph Tokenization for Transformers","created":"2026-05-21","cited_by_count":2,"area":null},{"id":"2502.06038","title":"Provably Overwhelming Transformer Models with Designed Inputs","created":"2025-02-09","cited_by_count":1,"area":null},{"id":"2503.04848","title":"Three tiers of computation in transformers and in brain architectures","created":"2025-03-05","cited_by_count":1,"area":null},{"id":"2505.17282","title":"Attention with Trained Embeddings Provably Selects Important Tokens","created":"2025-05-22","cited_by_count":1,"area":null},{"id":"2506.18739","title":"On the Existence of Universal Simulators of Attention","created":"2025-06-23","cited_by_count":1,"area":null},{"id":"2602.01763","title":"A Provable Expressiveness Hierarchy in Hybrid Linear-Full Attention","created":"2026-02-02","cited_by_count":1,"area":null},{"id":"2602.02707","title":"Every Bit Counts: A Theoretical Study of Precision-Expressivity Tradeoffs in Quantized Transformers","created":"2026-02-02","cited_by_count":1,"area":null},{"id":"2602.21371","title":"Interleaved Head Attention","created":"2026-02-24","cited_by_count":1,"area":null},{"id":"2605.12426","title":"Geometric Factual Recall in Transformers","created":"2026-05-12","cited_by_count":1,"area":null},{"id":"2608.04243","title":"Attention-based representations for multi-task computation","created":"2026-08-04","cited_by_count":1,"area":null},{"id":"2410.09867","title":"Towards characterizing the value of edge embeddings in Graph Neural Networks","created":"2024-10-13","cited_by_count":0,"area":null},{"id":"2509.07963","title":"Customizing the Inductive Biases of Softmax Attention using Structured Matrices","created":"2025-09-09","cited_by_count":0,"area":null},{"id":"2510.18638","title":"Optimality and NP-Hardness of Transformers in Learning Markovian Dynamical Functions","created":"2025-10-21","cited_by_count":0,"area":null},{"id":"2512.15600","title":"How Smoothing is N-simplicial Attention?","created":"2025-12-17","cited_by_count":0,"area":null},{"id":"2601.04563","title":"A Vision for Multisensory Intelligence: Sensing, Science, and Synergy","created":"2026-01-08","cited_by_count":0,"area":null}],"next_cursor":"100"}