{"list":"other","total":53,"rows":[{"id":"1706.03762","title":"Attention Is All You Need","created":"2017-06-12","cited_by_count":60361},{"id":"2104.09864","title":"RoFormer: Enhanced Transformer with Rotary Position Embedding","created":"2021-04-20","cited_by_count":4703},{"id":"1502.01852","title":"Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification","created":"2015-02-06","cited_by_count":3611},{"id":"1609.07843","title":"Pointer Sentinel Mixture Models","created":"2016-09-26","cited_by_count":3096},{"id":"2307.08691","title":"FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning","created":"2023-07-17","cited_by_count":2598},{"id":"2111.00396","title":"Efficiently Modeling Long Sequences with Structured State Spaces","created":"2021-10-31","cited_by_count":2465},{"id":"2006.16236","title":"Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention","created":"2020-06-29","cited_by_count":2022},{"id":"2002.05202","title":"GLU Variants Improve Transformer","created":"2020-02-12","cited_by_count":1804},{"id":"2405.21060","title":"Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality","created":"2024-05-31","cited_by_count":1336},{"id":"1409.1259","title":"On the Properties of Neural Machine Translation: Encoder-Decoder Approaches","created":"2014-09-03","cited_by_count":1304},{"id":"2108.12409","title":"Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation","created":"2021-08-27","cited_by_count":1103},{"id":"2401.10166","title":"VMamba: Visual State Space Model","created":"2024-01-18","cited_by_count":969},{"id":"2208.04933","title":"Simplified State Space Layers for Sequence Modeling","created":"2022-08-09","cited_by_count":788},{"id":"2212.07677","title":"Transformers learn in-context by gradient descent","created":"2022-12-15","cited_by_count":732},{"id":"2011.04006","title":"Long Range Arena: A Benchmark for Efficient Transformers","created":"2020-11-08","cited_by_count":700},{"id":"2307.08621","title":"Retentive Network: A Successor to Transformer for Large Language Models","created":"2023-07-17","cited_by_count":613},{"id":"2008.07669","title":"HiPPO: Recurrent Memory with Optimal Polynomial Projections","created":"2020-08-17","cited_by_count":605},{"id":"2412.06464","title":"Gated Delta Networks: Improving Mamba2 with Delta Rule","created":"2024-12-09","cited_by_count":532},{"id":"2312.06635","title":"Gated Linear Attention Transformers with Hardware-Efficient Training","created":"2023-12-11","cited_by_count":514},{"id":"2102.11174","title":"Linear Transformers Are Secretly Fast Weight Programmers","created":"2021-02-22","cited_by_count":502},{"id":"2212.14052","title":"Hungry Hungry Hippos: Towards Language Modeling with State Space Models","created":"2022-12-28","cited_by_count":499},{"id":"2206.11893","title":"On the Parameterization and Initialization of Diagonal State Space Models","created":"2022-06-23","cited_by_count":424},{"id":"2303.06349","title":"Resurrecting Recurrent Neural Networks for Long Sequences","created":"2023-03-11","cited_by_count":408},{"id":"1912.10077","title":"Are Transformers universal approximators of sequence-to-sequence functions?","created":"2019-12-20","cited_by_count":406},{"id":"2406.06484","title":"Parallelizing Linear Transformers with the Delta Rule over Sequence Length","created":"2024-06-10","cited_by_count":396},{"id":"2103.02143","title":"Random Feature Attention","created":"2021-03-03","cited_by_count":354},{"id":"2405.04517","title":"xLSTM: Extended Long Short-Term Memory","created":"2024-05-07","cited_by_count":327},{"id":"2402.19427","title":"Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models","created":"2024-02-29","cited_by_count":242},{"id":"2402.01032","title":"Repeat After Me: Transformers are Better than State Space Models at Copying","created":"2024-02-01","cited_by_count":216},{"id":"1908.11775","title":"Transformer Dissection: A Unified Understanding of Transformer's Attention via the Lens of Kernel","created":"2019-08-30","cited_by_count":213},{"id":"2402.18668","title":"Simple linear attention language models balance the recall-throughput tradeoff","created":"2024-02-28","cited_by_count":208},{"id":"2312.04927","title":"Zoology: Measuring and Improving Recall in Efficient Language Models","created":"2023-12-08","cited_by_count":204},{"id":"2306.15794","title":"HyenaDNA: Long-Range Genomic Sequence Modeling at Single Nucleotide Resolution","created":"2023-06-27","cited_by_count":194},{"id":"2404.08819","title":"The Illusion of State in State-Space Models","created":"2024-04-12","cited_by_count":187},{"id":"2406.07887","title":"An Empirical Study of Mamba-based Language Models","created":"2024-06-12","cited_by_count":186},{"id":"2403.03234","title":"Caduceus: Bi-Directional Equivariant Long-Range DNA Sequence Modeling","created":"2024-03-05","cited_by_count":113},{"id":"2508.14444","title":"NVIDIA Nemotron Nano 2: An Accurate and Efficient Hybrid Mamba-Transformer Reasoning Model","created":"2025-08-20","cited_by_count":108},{"id":"2403.01590","title":"The Hidden Attention of Mamba Models","created":"2024-03-03","cited_by_count":91},{"id":"2502.10297","title":"DeltaProduct: Improving State-Tracking in Linear RNNs via Householder Products","created":"2025-02-14","cited_by_count":75},{"id":"2402.19047","title":"Theoretical Foundations of Deep Selective State-Space Models","created":"2024-02-29","cited_by_count":74},{"id":"2403.17844","title":"Mechanistic Design and Scaling of Hybrid Architectures","created":"2024-03-26","cited_by_count":59},{"id":"1703.03906","title":"Massive Exploration of Neural Machine Translation Architectures","created":"2017-03-11","cited_by_count":55},{"id":"2503.02130","title":"Forgetting Transformer: Softmax Attention with a Forget Gate","created":"2025-03-03","cited_by_count":53},{"id":"2212.10544","title":"Pretraining Without Attention","created":"2022-12-20","cited_by_count":51},{"id":"2506.04761","title":"Log-Linear Attention","created":"2025-06-05","cited_by_count":40},{"id":"2307.11888","title":"Universality of Linear Recurrences Followed by Non-linear Projections: Finite-Width Guarantees and Benefits of Complex Eigenvalues","created":"2023-07-21","cited_by_count":36},{"id":"2505.16381","title":"PaTH Attention: Position Encoding via Accumulating Householder Transformations","created":"2025-05-22","cited_by_count":34},{"id":"2405.15731","title":"Understanding the differences in Foundation Models: Attention, State Space Models, and Recurrent Neural Networks","created":"2024-05-24","cited_by_count":26},{"id":"2405.21064","title":"Recurrent neural networks: vanishing and exploding gradients are not the end of the story","created":"2024-05-31","cited_by_count":25},{"id":"2005.09561","title":"Normalized Attention Without Probability Cage","created":"2020-05-19","cited_by_count":20},{"id":"2508.19029","title":"Revisiting associative recall in modern recurrent models","created":"2025-08-26","cited_by_count":8},{"id":"2402.00236","title":"Positional Encoding Helps Recurrent Neural Networks Handle a Large Vocabulary","created":"2024-01-31","cited_by_count":5},{"id":"2410.15578","title":"Generalized Probabilistic Attention Mechanism in Transformers","created":"2024-10-21","cited_by_count":2}],"next_cursor":null}