{"list":"other","total":52,"rows":[{"id":"1706.03762","title":"Attention Is All You Need","created":"2017-06-12","cited_by_count":60361},{"id":"2005.14165","title":"Language Models are Few-Shot Learners","created":"2020-05-28","cited_by_count":32419},{"id":"2302.13971","title":"LLaMA: Open and Efficient Foundation Language Models","created":"2023-02-27","cited_by_count":14132},{"id":"2312.00752","title":"Mamba: Linear-Time Sequence Modeling with Selective State Spaces","created":"2023-12-01","cited_by_count":4718},{"id":"2108.07258","title":"On the Opportunities and Risks of Foundation Models","created":"2021-08-16","cited_by_count":4033},{"id":"2307.08691","title":"FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning","created":"2023-07-17","cited_by_count":2598},{"id":"2111.00396","title":"Efficiently Modeling Long Sequences with Structured State Spaces","created":"2021-10-31","cited_by_count":2465},{"id":"2006.16236","title":"Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention","created":"2020-06-29","cited_by_count":2022},{"id":"2304.01373","title":"Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling","created":"2023-04-03","cited_by_count":1786},{"id":"2009.14794","title":"Rethinking Attention with Performers","created":"2020-09-30","cited_by_count":1601},{"id":"2401.09417","title":"Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model","created":"2024-01-17","cited_by_count":1056},{"id":"2401.10166","title":"VMamba: Visual State Space Model","created":"2024-01-18","cited_by_count":969},{"id":"2209.11895","title":"In-context Learning and Induction Heads","created":"2022-09-24","cited_by_count":964},{"id":"2305.13048","title":"RWKV: Reinventing RNNs for the Transformer Era","created":"2023-05-22","cited_by_count":827},{"id":"2208.04933","title":"Simplified State Space Layers for Sequence Modeling","created":"2022-08-09","cited_by_count":788},{"id":"2011.04006","title":"Long Range Arena: A Benchmark for Efficient Transformers","created":"2020-11-08","cited_by_count":700},{"id":"2307.08621","title":"Retentive Network: A Successor to Transformer for Large Language Models","created":"2023-07-17","cited_by_count":613},{"id":"2008.07669","title":"HiPPO: Recurrent Memory with Optimal Polynomial Projections","created":"2020-08-17","cited_by_count":605},{"id":"2312.06635","title":"Gated Linear Attention Transformers with Hardware-Efficient Training","created":"2023-12-11","cited_by_count":514},{"id":"2102.11174","title":"Linear Transformers Are Secretly Fast Weight Programmers","created":"2021-02-22","cited_by_count":502},{"id":"2212.14052","title":"Hungry Hungry Hippos: Towards Language Modeling with State Space Models","created":"2022-12-28","cited_by_count":499},{"id":"2206.11893","title":"On the Parameterization and Initialization of Diagonal State Space Models","created":"2022-06-23","cited_by_count":424},{"id":"2204.07118","title":"DeiT III: Revenge of the ViT","created":"2022-04-14","cited_by_count":419},{"id":"2303.06349","title":"Resurrecting Recurrent Neural Networks for Long Sequences","created":"2023-03-11","cited_by_count":408},{"id":"2302.10866","title":"Hyena Hierarchy: Towards Larger Convolutional Language Models","created":"2023-02-21","cited_by_count":393},{"id":"2203.14343","title":"Diagonal State Spaces are as Effective as Structured State Spaces","created":"2022-03-27","cited_by_count":350},{"id":"2405.04517","title":"xLSTM: Extended Long Short-Term Memory","created":"2024-05-07","cited_by_count":327},{"id":"2402.19427","title":"Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models","created":"2024-02-29","cited_by_count":242},{"id":"1908.11775","title":"Transformer Dissection: A Unified Understanding of Transformer's Attention via the Lens of Kernel","created":"2019-08-30","cited_by_count":213},{"id":"2402.18668","title":"Simple linear attention language models balance the recall-throughput tradeoff","created":"2024-02-28","cited_by_count":208},{"id":"2312.04927","title":"Zoology: Measuring and Improving Recall in Efficient Language Models","created":"2023-12-08","cited_by_count":204},{"id":"2404.08819","title":"The Illusion of State in State-Space Models","created":"2024-04-12","cited_by_count":187},{"id":"2202.09729","title":"It's Raw! Audio Generation with State-Space Models","created":"2022-02-20","cited_by_count":171},{"id":"2404.07904","title":"HGRN2: Gated Linear RNNs with State Expansion","created":"2024-04-11","cited_by_count":127},{"id":"2401.06104","title":"Transformers are Multi-State RNNs","created":"2024-01-11","cited_by_count":120},{"id":"2403.03234","title":"Caduceus: Bi-Directional Equivariant Long-Range DNA Sequence Modeling","created":"2024-03-05","cited_by_count":113},{"id":"2303.03982","title":"Structured State Space Models for In-Context Reinforcement Learning","created":"2023-03-07","cited_by_count":111},{"id":"2403.01590","title":"The Hidden Attention of Mamba Models","created":"2024-03-03","cited_by_count":91},{"id":"2210.09298","title":"What Makes Convolutional Models Great on Long Sequence Modeling?","created":"2022-10-17","cited_by_count":87},{"id":"2402.19047","title":"Theoretical Foundations of Deep Selective State-Space Models","created":"2024-02-29","cited_by_count":74},{"id":"2403.06467","title":"Point Mamba: A Novel Point Cloud Backbone Based on State Space Model with Octree-Based Ordering Strategy","created":"2024-03-11","cited_by_count":73},{"id":"2401.13660","title":"MambaByte: Token-free Selective State Space Model","created":"2024-01-24","cited_by_count":59},{"id":"2212.10544","title":"Pretraining Without Attention","created":"2022-12-20","cited_by_count":51},{"id":"2311.01927","title":"GateLoop: Fully Data-Controlled Linear Recurrence for Sequence Modeling","created":"2023-11-03","cited_by_count":46},{"id":"2210.06583","title":"S4ND: Modeling Images and Videos as Multidimensional Signals Using State Spaces","created":"2022-10-12","cited_by_count":41},{"id":"2309.13414","title":"State-space Models with Layer-wise Nonlinearity are Universal Approximators with Exponential Decaying Memory","created":"2023-09-23","cited_by_count":39},{"id":"2307.11888","title":"Universality of Linear Recurrences Followed by Non-linear Projections: Finite-Width Guarantees and Benefits of Complex Eigenvalues","created":"2023-07-21","cited_by_count":36},{"id":"2403.16899","title":"State Space Models as Foundation Models: A Control Theoretic Overview","created":"2024-03-25","cited_by_count":19},{"id":"2309.11197","title":"The Languini Kitchen: Enabling Language Modelling Research at Different Scales of Compute","created":"2023-09-20","cited_by_count":10},{"id":"2309.01775","title":"Gated recurrent neural networks discover attention","created":"2023-09-04","cited_by_count":7},{"id":"2401.14029","title":"Towards a Systems Theory of Algorithms","created":"2024-01-25","cited_by_count":7},{"id":"2403.02920","title":"TaylorShift: Shifting the Complexity of Self-Attention from Squared to Linear (and Back) using Taylor-Softmax","created":"2024-03-05","cited_by_count":7}],"next_cursor":null}