{"list":"other","total":66,"rows":[{"id":"1706.03762","title":"Attention Is All You Need","created":"2017-06-12","cited_by_count":60361},{"id":"2005.14165","title":"Language Models are Few-Shot Learners","created":"2020-05-28","cited_by_count":32419},{"id":"2312.00752","title":"Mamba: Linear-Time Sequence Modeling with Selective State Spaces","created":"2023-12-01","cited_by_count":4718},{"id":"2111.00396","title":"Efficiently Modeling Long Sequences with Structured State Spaces","created":"2021-10-31","cited_by_count":2465},{"id":"2006.16236","title":"Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention","created":"2020-06-29","cited_by_count":2022},{"id":"2202.12837","title":"Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?","created":"2022-02-25","cited_by_count":1434},{"id":"2405.21060","title":"Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality","created":"2024-05-31","cited_by_count":1336},{"id":"2111.02080","title":"An Explanation of In-context Learning as Implicit Bayesian Inference","created":"2021-11-03","cited_by_count":938},{"id":"2305.13048","title":"RWKV: Reinventing RNNs for the Transformer Era","created":"2023-05-22","cited_by_count":827},{"id":"2212.07677","title":"Transformers learn in-context by gradient descent","created":"2022-12-15","cited_by_count":732},{"id":"2208.01066","title":"What Can Transformers Learn In-Context? A Case Study of Simple Function Classes","created":"2022-08-01","cited_by_count":715},{"id":"2211.15661","title":"What learning algorithm is in-context learning? Investigations with linear models","created":"2022-11-28","cited_by_count":684},{"id":"2307.08621","title":"Retentive Network: A Successor to Transformer for Large Language Models","created":"2023-07-17","cited_by_count":613},{"id":"2312.06635","title":"Gated Linear Attention Transformers with Hardware-Efficient Training","created":"2023-12-11","cited_by_count":514},{"id":"2303.06349","title":"Resurrecting Recurrent Neural Networks for Long Sequences","created":"2023-03-11","cited_by_count":408},{"id":"2103.02143","title":"Random Feature Attention","created":"2021-03-03","cited_by_count":354},{"id":"2306.09927","title":"Trained Transformers Learn Linear Models In-Context","created":"2023-06-16","cited_by_count":337},{"id":"2405.04517","title":"xLSTM: Extended Long Short-Term Memory","created":"2024-05-07","cited_by_count":327},{"id":"2306.00297","title":"Transformers learn to implement preconditioned gradient descent for in-context learning","created":"2023-06-01","cited_by_count":279},{"id":"2301.07067","title":"Transformers as Algorithms: Generalization and Stability in In-context Learning","created":"2023-01-17","cited_by_count":222},{"id":"2402.01032","title":"Repeat After Me: Transformers are Better than State Space Models at Copying","created":"2024-02-01","cited_by_count":216},{"id":"2402.18668","title":"Simple linear attention language models balance the recall-throughput tradeoff","created":"2024-02-28","cited_by_count":208},{"id":"2307.03576","title":"One Step of Gradient Descent is Provably the Optimal In-Context Learner with One Layer of Linear Self-Attention","created":"2023-07-07","cited_by_count":170},{"id":"2404.05892","title":"Eagle and Finch: RWKV with Matrix-Valued States and Dynamic Recurrence","created":"2024-04-08","cited_by_count":159},{"id":"2405.05254","title":"You Only Cache Once: Decoder-Decoder Architectures for Language Models","created":"2024-05-08","cited_by_count":144},{"id":"2404.07904","title":"HGRN2: Gated Linear RNNs with State Expansion","created":"2024-04-11","cited_by_count":127},{"id":"2310.05249","title":"In-Context Convergence of Transformers","created":"2023-10-08","cited_by_count":115},{"id":"2402.04248","title":"Can Mamba Learn How to Learn? A Comparative Study on In-Context Learning Tasks","created":"2024-02-06","cited_by_count":105},{"id":"2309.05858","title":"Uncovering mesa-optimization algorithms in Transformers","created":"2023-09-11","cited_by_count":100},{"id":"2310.08391","title":"How Many Pretraining Tasks Are Needed for In-Context Learning of Linear Regression?","created":"2023-10-12","cited_by_count":96},{"id":"2210.09221","title":"Vision Transformers provably learn spatial structure","created":"2022-10-13","cited_by_count":93},{"id":"2205.11961","title":"ATTEMPT: Parameter-Efficient Multi-task Tuning via Attentional Mixtures of Soft Prompts","created":"2022-05-24","cited_by_count":85},{"id":"2302.06015","title":"A Theoretical Understanding of Shallow Vision Transformers: Learning, Generalization, and Sample Complexity","created":"2023-02-12","cited_by_count":77},{"id":"2308.16898","title":"Transformers as Support Vector Machines","created":"2023-08-31","cited_by_count":74},{"id":"2402.19442","title":"Training Dynamics of Multi-Head Softmax Attention for In-Context Learning: Emergence, Convergence, and Optimality","created":"2024-02-29","cited_by_count":74},{"id":"2410.08292","title":"Can Looped Transformers Learn to Implement Multi-step Gradient Descent for In-context Learning?","created":"2024-10-10","cited_by_count":65},{"id":"2306.13596","title":"Max-Margin Token Selection in Attention Mechanism","created":"2023-06-23","cited_by_count":58},{"id":"2310.00535","title":"JoMA: Demystifying Multilayer Transformers via JOint Dynamics of MLP and Attention","created":"2023-10-01","cited_by_count":52},{"id":"2306.03435","title":"On the Role of Attention in Prompt-tuning","created":"2023-06-06","cited_by_count":50},{"id":"2311.01927","title":"GateLoop: Fully Data-Controlled Linear Recurrence for Sequence Modeling","created":"2023-11-03","cited_by_count":46},{"id":"2402.03170","title":"Is Mamba Capable of In-Context Learning?","created":"2024-02-05","cited_by_count":45},{"id":"2402.18819","title":"Dual Operating Modes of In-Context Learning","created":"2024-02-29","cited_by_count":43},{"id":"2401.15530","title":"An Information-Theoretic Analysis of In-Context Learning","created":"2024-01-28","cited_by_count":40},{"id":"2402.04161","title":"Attention with Markov: A Framework for Principled Analysis of Transformers via Markov Chains","created":"2024-02-06","cited_by_count":40},{"id":"2310.12680","title":"On the Optimization and Generalization of Multi-head Attention","created":"2023-10-19","cited_by_count":37},{"id":"2402.13512","title":"From Self-Attention to Markov Models: Unveiling the Dynamics of Generative Transformers","created":"2024-02-21","cited_by_count":34},{"id":"2403.08081","title":"Mechanics of Next Token Prediction with Self-Attention","created":"2024-03-12","cited_by_count":34},{"id":"2402.05738","title":"Implicit Bias and Fast Convergence Rates for Self-attention","created":"2024-02-08","cited_by_count":31},{"id":"2205.08078","title":"Unraveling Attention via Convex Duality: Analysis and Interpretations of Vision Transformers","created":"2022-05-17","cited_by_count":30},{"id":"2308.06912","title":"CausalLM is not optimal for in-context learning","created":"2023-08-14","cited_by_count":26},{"id":"2407.10005","title":"Fine-grained Analysis of In-context Linear Estimation: Data, Architecture, and Beyond","created":"2024-07-13","cited_by_count":24},{"id":"2408.15417","title":"Implicit Geometry of Next-token Prediction: From Language Sparsity Patterns to Model Representations","created":"2024-08-27","cited_by_count":24},{"id":"2402.11639","title":"In-Context Learning with Transformers: Softmax Attention Adapts to Function Lipschitzness","created":"2024-02-18","cited_by_count":22},{"id":"2406.06893","title":"Transformers Provably Learn Sparse Token Selection While Fully-Connected Nets Cannot","created":"2024-06-11","cited_by_count":22},{"id":"2402.04084","title":"Provably learning a multi-head attention layer","created":"2024-02-06","cited_by_count":21},{"id":"2402.02098","title":"Self-attention Networks Localize When QK-eigenspectrum Concentrates","created":"2024-02-03","cited_by_count":19},{"id":"2403.08699","title":"Implicit Regularization of Gradient Flow on One-Layer Softmax Attention","created":"2024-03-13","cited_by_count":19},{"id":"2407.00256","title":"One Prompt is not Enough: Automated Construction of a Mixture-of-Expert Prompts","created":"2024-06-28","cited_by_count":16},{"id":"2312.07887","title":"Learn or Recall? Revisiting Incremental Learning with Pre-trained Language Models","created":"2023-12-13","cited_by_count":13},{"id":"2409.17335","title":"Non-asymptotic Convergence of Training Transformers for Next-token Prediction","created":"2024-09-25","cited_by_count":13},{"id":"2211.11052","title":"Convexifying Transformers: Improving optimization and understanding of transformer networks","created":"2022-11-20","cited_by_count":12},{"id":"2310.02842","title":"Sweeping Heterogeneity with Smart MoPs: Mixture of Prompts for LLM Task Adaptation","created":"2023-10-04","cited_by_count":12},{"id":"2310.12462","title":"Unmasking Transformers: A Theoretical Approach to Data Recovery via Attention Weights","created":"2023-10-19","cited_by_count":12},{"id":"2410.14581","title":"Optimizing Attention with Mirror Descent: Generalized Max-Margin Token Selection","created":"2024-10-18","cited_by_count":9},{"id":"2501.18187","title":"On the Role of Transformer Feed-Forward Layers in Nonlinear In-Context Learning","created":"2025-01-30","cited_by_count":7},{"id":"2406.06594","title":"Stock Movement Prediction with Multimodal Stable Fusion via Gated Cross-Attention Mechanism","created":"2024-06-06","cited_by_count":3}],"next_cursor":null}