{"total":121,"rows":[{"id":"2307.05782","title":"Large Language Models","created":"2023-07-11","cited_by_count":780,"area":null},{"id":"2208.01066","title":"What Can Transformers Learn In-Context? A Case Study of Simple Function Classes","created":"2022-08-01","cited_by_count":715,"area":null},{"id":"2212.14052","title":"Hungry Hungry Hippos: Towards Language Modeling with State Space Models","created":"2022-12-28","cited_by_count":499,"area":null},{"id":"2306.09927","title":"Trained Transformers Learn Linear Models In-Context","created":"2023-06-16","cited_by_count":337,"area":null},{"id":"2306.00297","title":"Transformers learn to implement preconditioned gradient descent for in-context learning","created":"2023-06-01","cited_by_count":279,"area":null},{"id":"2210.10749","title":"Transformers Learn Shortcuts to Automata","created":"2022-10-19","cited_by_count":268,"area":null},{"id":"2402.01032","title":"Repeat After Me: Transformers are Better than State Space Models at Copying","created":"2024-02-01","cited_by_count":216,"area":null},{"id":"2207.08799","title":"Hidden Progress in Deep Learning: SGD Learns Parities Near the Computational Limit","created":"2022-07-18","cited_by_count":187,"area":null},{"id":"2306.00802","title":"Birth of a Transformer: A Memory Viewpoint","created":"2023-06-01","cited_by_count":161,"area":null},{"id":"2306.02896","title":"Representational Strengths and Limitations of Transformers","created":"2023-06-05","cited_by_count":133,"area":null},{"id":"2305.19420","title":"What and How does In-Context Learning Learn? Bayesian Model Averaging, Parameterization, and Generalization","created":"2023-05-30","cited_by_count":118,"area":null},{"id":"2305.16380","title":"Scan and Snap: Understanding Training Dynamics and Token Composition in 1-layer Transformer","created":"2023-05-25","cited_by_count":102,"area":null},{"id":"2210.09221","title":"Vision Transformers provably learn spatial structure","created":"2022-10-13","cited_by_count":93,"area":null},{"id":"2303.04245","title":"How Do Transformers Learn Topic Structure: Towards a Mechanistic Understanding","created":"2023-03-07","cited_by_count":86,"area":null},{"id":"2302.06015","title":"A Theoretical Understanding of Shallow Vision Transformers: Learning, Generalization, and Sample Complexity","created":"2023-02-12","cited_by_count":77,"area":null},{"id":"2306.00946","title":"Exposing Attention Glitches with Flip-Flop Language Modeling","created":"2023-06-01","cited_by_count":75,"area":null},{"id":"2308.16898","title":"Transformers as Support Vector Machines","created":"2023-08-31","cited_by_count":74,"area":null},{"id":"2402.09963","title":"Why are Sensitive Functions Hard for Transformers?","created":"2024-02-15","cited_by_count":74,"area":null},{"id":"2207.09238","title":"Formal Algorithms for Transformers","created":"2022-07-19","cited_by_count":72,"area":null},{"id":"2403.04797","title":"Found in the Middle: How Language Models Use Long Contexts Better via Plug-and-Play Positional Encoding","created":"2024-03-05","cited_by_count":72,"area":null},{"id":"2211.12316","title":"Simplicity Bias in Transformers and their Ability to Learn Sparse Boolean Functions","created":"2022-11-22","cited_by_count":70,"area":null},{"id":"2306.13596","title":"Max-Margin Token Selection in Attention Mechanism","created":"2023-06-23","cited_by_count":58,"area":null},{"id":"2310.00535","title":"JoMA: Demystifying Multilayer Transformers via JOint Dynamics of MLP and Attention","created":"2023-10-01","cited_by_count":52,"area":null},{"id":"2306.03435","title":"On the Role of Attention in Prompt-tuning","created":"2023-06-06","cited_by_count":50,"area":null},{"id":"2303.08117","title":"Do Transformers Parse while Predicting the Masked Word?","created":"2023-03-14","cited_by_count":48,"area":null},{"id":"2307.14023","title":"Are Transformers with One Layer Self-Attention Using Low-Rank Weight Matrices Universal Approximators?","created":"2023-07-26","cited_by_count":47,"area":null},{"id":"2303.16504","title":"An Over-parameterized Exponential Regression","created":"2023-03-29","cited_by_count":45,"area":null},{"id":"2303.15725","title":"Solving Regularized Exp, Cosh and Sinh Regression Problems","created":"2023-03-28","cited_by_count":44,"area":null},{"id":"2304.13276","title":"The Closeness of In-Context Learning and Weight Shifting for Softmax Regression","created":"2023-04-26","cited_by_count":41,"area":null},{"id":"2306.02010","title":"Memorization Capacity of Multi-Head Attention in Transformers","created":"2023-06-03","cited_by_count":40,"area":null},{"id":"2307.11353","title":"What can a Single Attention Layer Learn? A Study Through the Random Features Lens","created":"2023-07-21","cited_by_count":40,"area":null},{"id":"2401.15530","title":"An Information-Theoretic Analysis of In-Context Learning","created":"2024-01-28","cited_by_count":40,"area":null},{"id":"2404.03828","title":"Outlier-Efficient Hopfield Layers for Large Transformer-Based Models","created":"2024-04-04","cited_by_count":38,"area":null},{"id":"2408.10147","title":"In-Context Learning with Representations: Contextual Generalization of Trained Transformers","created":"2024-08-19","cited_by_count":38,"area":null},{"id":"2310.12680","title":"On the Optimization and Generalization of Multi-head Attention","created":"2023-10-19","cited_by_count":37,"area":null},{"id":"2410.02724","title":"Large Language Models as Markov Chains","created":"2024-10-03","cited_by_count":35,"area":null},{"id":"2402.13512","title":"From Self-Attention to Markov Models: Unveiling the Dynamics of Generative Transformers","created":"2024-02-21","cited_by_count":34,"area":null},{"id":"2403.08081","title":"Mechanics of Next Token Prediction with Self-Attention","created":"2024-03-12","cited_by_count":34,"area":null},{"id":"2305.18699","title":"Approximation and Estimation Ability of Transformers for Sequence-to-Sequence Functions with Infinite Dimensional Input","created":"2023-05-30","cited_by_count":32,"area":null},{"id":"2312.01429","title":"Transformers are uninterpretable with myopic methods: a case study with bounded Dyck grammars","created":"2023-12-03","cited_by_count":30,"area":null},{"id":"2402.03902","title":"A phase transition between positional and semantic learning in a solvable model of dot-product attention","created":"2024-02-06","cited_by_count":30,"area":null},{"id":"2411.06646","title":"Understanding Scaling Laws with Statistical and Approximation Theory for Transformer Neural Networks on Intrinsically Low-dimensional Data","created":"2024-11-11","cited_by_count":30,"area":null},{"id":"2306.02295","title":"A Mathematical Abstraction for Balancing the Trade-off Between Creativity and Reality in Large Language Models","created":"2023-06-04","cited_by_count":29,"area":null},{"id":"2310.13088","title":"Sequence Length Independent Norm-Based Generalization Bounds for Transformers","created":"2023-10-19","cited_by_count":29,"area":null},{"id":"2411.04551","title":"Measure-to-measure interpolation using Transformers","created":"2024-11-07","cited_by_count":29,"area":null},{"id":"2305.18475","title":"Approximation Rate of the Transformer Architecture for Sequence Modeling","created":"2023-05-29","cited_by_count":28,"area":null},{"id":"2402.04084","title":"Provably learning a multi-head attention layer","created":"2024-02-06","cited_by_count":21,"area":null},{"id":"2304.14329","title":"Learning to Extrapolate: A Transductive Approach","created":"2023-04-27","cited_by_count":19,"area":null},{"id":"2306.13945","title":"Large Sequence Models for Sequential Decision-Making: A Survey","created":"2023-06-24","cited_by_count":19,"area":null},{"id":"2307.01189","title":"Trainable Transformer in Transformer","created":"2023-07-03","cited_by_count":17,"area":null},{"id":"2212.14852","title":"An Analysis of Attention via the Lens of Exchangeability and Latent Variable Models","created":"2022-12-30","cited_by_count":16,"area":null},{"id":"2309.05217","title":"Quantifying and Attributing the Hallucination of Large Language Models via Association Analysis","created":"2023-09-11","cited_by_count":15,"area":null},{"id":"2310.11685","title":"Why Softmax Attention Outperforms Linear Attention","created":"2023-10-18","cited_by_count":15,"area":null},{"id":"2412.09925","title":"Simulating Hard Attention Using Soft Attention","created":"2024-12-13","cited_by_count":15,"area":null},{"id":"2310.11428","title":"Butterfly Effects of SGD Noise: Error Amplification in Behavior Cloning and Autoregression","created":"2023-10-17","cited_by_count":14,"area":null},{"id":"2402.18551","title":"Implicit Optimization Bias of Next-Token Prediction in Linear Models","created":"2024-02-28","cited_by_count":14,"area":null},{"id":"2311.01575","title":"On the Convergence of Encoder-only Shallow Transformers","created":"2023-11-02","cited_by_count":13,"area":null},{"id":"2409.17335","title":"Non-asymptotic Convergence of Training Transformers for Next-token Prediction","created":"2024-09-25","cited_by_count":13,"area":null},{"id":"2209.09845","title":"Relational Reasoning via Set Transformers: Provable Efficiency and Applications to MARL","created":"2022-09-20","cited_by_count":12,"area":null},{"id":"2402.03485","title":"Attention Meets Post-hoc Interpretability: A Mathematical Perspective","created":"2024-02-05","cited_by_count":11,"area":null},{"id":"2510.05184","title":"Representation Potentials of Foundation Models for Multimodal Alignment: A Survey","created":"2025-10-05","cited_by_count":11,"area":null},{"id":"2409.17677","title":"On the Optimal Memorization Capacity of Transformers","created":"2024-09-26","cited_by_count":10,"area":null},{"id":"2306.06203","title":"FLSL: Feature-level Self-supervised Learning","created":"2023-06-09","cited_by_count":9,"area":null},{"id":"2405.15926","title":"Dissecting the Interplay of Attention Paths in a Statistical Mechanics Theory of Transformers","created":"2024-05-24","cited_by_count":9,"area":null},{"id":"2410.09605","title":"Training Dynamics of Transformers to Recognize Word Co-occurrence via Gradient Flow Analysis","created":"2024-10-12","cited_by_count":9,"area":null},{"id":"2309.06054","title":"Breaking through the learning plateaus of in-context learning in Transformer","created":"2023-09-12","cited_by_count":8,"area":null},{"id":"2403.06871","title":"On the Generalization Ability of Unsupervised Pretraining","created":"2024-03-11","cited_by_count":8,"area":null},{"id":"2406.04090","title":"Interpretable Lightweight Transformer via Unrolling of Learned Graph Smoothness Priors","created":"2024-06-06","cited_by_count":8,"area":null},{"id":"2410.11474","title":"How Transformers Get Rich: Approximation and Dynamics Analysis","created":"2024-10-15","cited_by_count":8,"area":null},{"id":"2306.09520","title":"Ensembled Prediction Intervals for Causal Outcomes Under Hidden Confounding","created":"2023-06-15","cited_by_count":7,"area":null},{"id":"2501.18187","title":"On the Role of Transformer Feed-Forward Layers in Nonlinear In-Context Learning","created":"2025-01-30","cited_by_count":7,"area":null},{"id":"2302.08374","title":"Efficiency 360: Efficient Vision Transformers","created":"2023-02-16","cited_by_count":6,"area":null},{"id":"2407.21046","title":"Promises and Pitfalls of Generative Masked Language Modeling: Theoretical Framework and Practical Guidelines","created":"2024-07-22","cited_by_count":6,"area":null},{"id":"2506.05249","title":"On the Convergence of Gradient Descent on Learning Transformers with Residual Connections","created":"2025-06-05","cited_by_count":6,"area":null},{"id":"2510.00368","title":"The Transformer Cookbook","created":"2025-10-01","cited_by_count":6,"area":null},{"id":"2402.02969","title":"Towards Understanding the Word Sensitivity of Attention Layers: A Study via Random Features","created":"2024-02-05","cited_by_count":5,"area":null},{"id":"2407.16153","title":"On the Benefits of Rank in Attention Layers","created":"2024-07-23","cited_by_count":5,"area":null},{"id":"2410.10101","title":"Learning Linear Attention in Polynomial Time","created":"2024-10-14","cited_by_count":5,"area":null},{"id":"2410.23610","title":"Global Convergence in Training Large-Scale Transformers","created":"2024-10-31","cited_by_count":5,"area":null},{"id":"2502.12187","title":"Hallucinations are inevitable but can be made statistically negligible","created":"2025-02-15","cited_by_count":5,"area":null},{"id":"2504.13558","title":"Transformers Can Overcome the Curse of Dimensionality: A Theoretical Study from an Approximation Perspective","created":"2025-04-18","cited_by_count":5,"area":null},{"id":"2505.03205","title":"Transformers for Learning on Noisy and Task-Level Manifolds: Approximation and Generalization Insights","created":"2025-05-06","cited_by_count":5,"area":null},{"id":"2410.11500","title":"On Generalisation Error Bounds for Transformers","created":"2024-10-15","cited_by_count":4,"area":null},{"id":"2504.12175","title":"Approximation Bounds for Transformer Networks with Application to Regression","created":"2025-04-16","cited_by_count":4,"area":null},{"id":"2210.06741","title":"Why self-attention is Natural for Sequence-to-Sequence Problems? A Perspective from Symmetries","created":"2022-10-13","cited_by_count":3,"area":null},{"id":"2503.11272","title":"When Do Transformers Outperform Feedforward and Recurrent Networks? A Statistical Perspective","created":"2025-03-14","cited_by_count":3,"area":null},{"id":"2602.18849","title":"Exact Attention Sensitivity and the Geometry of Transformer Stability","created":"2026-02-21","cited_by_count":3,"area":null},{"id":"2605.07097","title":"Every Feedforward Neural Network Definable in an o-Minimal Structure Has Finite Sample Complexity","created":"2026-05-08","cited_by_count":3,"area":null},{"id":"2209.00735","title":"Recurrent Convolutional Neural Networks Learn Succinct Learning Algorithms","created":"2022-09-01","cited_by_count":2,"area":null},{"id":"2401.11143","title":"Density Adaptive Attention is All You Need: Robust Parameter-Efficient Fine-Tuning Across Multiple Modalities","created":"2024-01-20","cited_by_count":2,"area":null},{"id":"2405.20278","title":"Length independent generalization bounds for deep SSM architectures via Rademacher contraction and stability constraints","created":"2024-05-30","cited_by_count":2,"area":null},{"id":"2512.09275","title":"Impact of Positional Encoding: Clean and Adversarial Rademacher Complexity for Transformers under In-Context Regression","created":"2025-12-10","cited_by_count":2,"area":null},{"id":"2601.06969","title":"Generalization Bounds for Transformer Channel Decoders","created":"2026-01-11","cited_by_count":2,"area":null},{"id":"2603.19954","title":"On the Ability of Transformers to Verify Plans","created":"2026-03-20","cited_by_count":2,"area":null},{"id":"2605.08811","title":"Learning Theory of Transformers: Local-to-Global Approximation via Softmax Partition of Unity","created":"2026-05-09","cited_by_count":2,"area":null},{"id":"2605.28075","title":"Measure-to-measure Regression with Transformers","created":"2026-05-27","cited_by_count":2,"area":null},{"id":"2606.09731","title":"Tight Sample Complexity of Transformers","created":"2026-06-08","cited_by_count":2,"area":null},{"id":"2407.10949","title":"Representing Rule-based Chatbots with Transformers","created":"2024-07-15","cited_by_count":1,"area":null},{"id":"2410.08421","title":"Generalizable autoregressive modeling of time series through functional narratives","created":"2024-10-10","cited_by_count":1,"area":null},{"id":"2506.18739","title":"On the Existence of Universal Simulators of Attention","created":"2025-06-23","cited_by_count":1,"area":null}],"next_cursor":"100"}