{"list":"other","total":53,"rows":[{"id":"1706.03762","title":"Attention Is All You Need","created":"2017-06-12","cited_by_count":60361},{"id":"2005.14165","title":"Language Models are Few-Shot Learners","created":"2020-05-28","cited_by_count":32419},{"id":"1910.10683","title":"Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer","created":"2019-10-23","cited_by_count":14599},{"id":"2104.09864","title":"RoFormer: Enhanced Transformer with Rotary Position Embedding","created":"2021-04-20","cited_by_count":4703},{"id":"1409.0473","title":"Neural Machine Translation by Jointly Learning to Align and Translate","created":"2014-09-01","cited_by_count":4548},{"id":"2205.14135","title":"FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness","created":"2022-05-27","cited_by_count":3673},{"id":"1609.07843","title":"Pointer Sentinel Mixture Models","created":"2016-09-26","cited_by_count":3096},{"id":"2307.08691","title":"FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning","created":"2023-07-17","cited_by_count":2598},{"id":"2303.12712","title":"Sparks of Artificial General Intelligence: Early experiments with GPT-4","created":"2023-03-22","cited_by_count":2592},{"id":"2202.03629","title":"Survey of Hallucination in Natural Language Generation","created":"2022-02-08","cited_by_count":2075},{"id":"1901.02860","title":"Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context","created":"2019-01-09","cited_by_count":1850},{"id":"2002.05202","title":"GLU Variants Improve Transformer","created":"2020-02-12","cited_by_count":1804},{"id":"2007.14062","title":"Big Bird: Transformers for Longer Sequences","created":"2020-07-28","cited_by_count":1684},{"id":"1904.10509","title":"Generating Long Sequences with Sparse Transformers","created":"2019-04-23","cited_by_count":1546},{"id":"1910.07467","title":"Root Mean Square Layer Normalization","created":"2019-10-16","cited_by_count":1480},{"id":"2404.06654","title":"RULER: What's the Real Context Size of Your Long-Context Language Models?","created":"2024-04-09","cited_by_count":1346},{"id":"2108.12409","title":"Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation","created":"2021-08-27","cited_by_count":1103},{"id":"1508.04025","title":"Effective Approaches to Attention-based Neural Machine Translation","created":"2015-08-17","cited_by_count":1026},{"id":"2410.05229","title":"GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Models","created":"2024-10-07","cited_by_count":550},{"id":"2303.06349","title":"Resurrecting Recurrent Neural Networks for Long Sequences","created":"2023-03-11","cited_by_count":408},{"id":"2305.19466","title":"The Impact of Positional Encoding on Length Generalization in Transformers","created":"2023-05-31","cited_by_count":327},{"id":"2210.10749","title":"Transformers Learn Shortcuts to Automata","created":"2022-10-19","cited_by_count":268},{"id":"2310.16028","title":"What Algorithms can Transformers Learn? A Study in Length Generalization","created":"2023-10-24","cited_by_count":199},{"id":"2404.08819","title":"The Illusion of State in State-Space Models","created":"2024-04-12","cited_by_count":187},{"id":"2106.06981","title":"Thinking Like Transformers","created":"2021-06-13","cited_by_count":184},{"id":"1905.05702","title":"Sparse Sequence-to-Sequence Models","created":"2019-05-14","cited_by_count":170},{"id":"2106.16213","title":"Saturated Transformers are Constant-Depth Threshold Circuits","created":"2021-06-30","cited_by_count":139},{"id":"1908.08962","title":"Well-Read Students Learn Better: On the Importance of Pre-training Compact Models","created":"2019-08-23","cited_by_count":120},{"id":"2305.16843","title":"Randomized Positional Encodings Boost Length Generalization of Transformers","created":"2023-05-26","cited_by_count":117},{"id":"2410.06205","title":"Round and Round We Go! What makes Rotary Positional Encodings useful?","created":"2024-10-08","cited_by_count":115},{"id":"2107.02192","title":"Long-Short Transformer: Efficient Transformers for Language and Vision","created":"2021-07-05","cited_by_count":102},{"id":"2210.03050","title":"State-of-the-art generalisation research in NLP: A taxonomy and review","created":"2022-10-06","cited_by_count":98},{"id":"2306.00946","title":"Exposing Attention Glitches with Flip-Flop Language Modeling","created":"2023-06-01","cited_by_count":75},{"id":"2110.07732","title":"The Neural Data Router: Adaptive Control Flow in Transformers Improves Systematic Generalization","created":"2021-10-14","cited_by_count":62},{"id":"2208.05309","title":"Looking for a Needle in a Haystack: A Comprehensive Study of Hallucinations in Neural Machine Translation","created":"2022-08-10","cited_by_count":61},{"id":"1711.02013","title":"Neural Language Modeling by Jointly Learning Syntax and Lexicon","created":"2017-11-02","cited_by_count":57},{"id":"2405.18719","title":"Contextual Position Encoding: Learning to Count What's Important","created":"2024-05-29","cited_by_count":54},{"id":"2503.02130","title":"Forgetting Transformer: Softmax Attention with a Forget Gate","created":"2025-03-03","cited_by_count":53},{"id":"2104.07012","title":"Sparse Attention with Linear Units","created":"2021-04-14","cited_by_count":41},{"id":"1911.03872","title":"Location Attention for Extrapolation to Longer Sequences","created":"2019-11-10","cited_by_count":33},{"id":"2310.01749","title":"Stack Attention: Improving the Ability of Transformers to Model Hierarchical Patterns","created":"2023-10-03","cited_by_count":17},{"id":"2106.06038","title":"Modeling Hierarchical Structures with Continuous Recursive Neural Networks","created":"2021-06-10","cited_by_count":15},{"id":"2406.08787","title":"A Survey on Compositional Learning of AI Models: Theoretical and Experimental Practices","created":"2024-06-13","cited_by_count":15},{"id":"2409.11250","title":"Linear Recency Bias During Training Improves Transformers' Fit to Reading Times","created":"2024-09-17","cited_by_count":14},{"id":"2408.05506","title":"Your Context Is Not an Array: Unveiling Random Access Limitations in Transformers","created":"2024-08-10","cited_by_count":11},{"id":"2002.09402","title":"Addressing Some Limitations of Transformers with Feedback Memory","created":"2020-02-21","cited_by_count":9},{"id":"2305.20019","title":"Monotonic Location Attention for Length Generalization","created":"2023-05-31","cited_by_count":9},{"id":"2410.17498","title":"Mechanisms of Symbol Processing for In-Context Learning in Transformer Networks","created":"2024-10-23","cited_by_count":6},{"id":"2311.00128","title":"On the effect of curriculum learning with developmental data for grammar acquisition","created":"2023-10-31","cited_by_count":5},{"id":"2412.14076","title":"Compositional Generalization Across Distributional Shifts with Sparse Tree Operations","created":"2024-12-18","cited_by_count":5},{"id":"2404.01860","title":"Self-StrAE at SemEval-2024 Task 1: Making Self-Structuring AutoEncoders Learn More With Less","created":"2024-04-02","cited_by_count":3},{"id":"2407.17771","title":"Banyan: Improved Representation Learning with Explicit Structure","created":"2024-07-25","cited_by_count":3},{"id":"2305.05588","title":"StrAE: Autoencoding for Pre-Trained Embeddings using Explicit Structure","created":"2023-05-09","cited_by_count":2}],"next_cursor":null}