{"list":"other","total":61,"rows":[{"id":"1706.03762","title":"Attention Is All You Need","created":"2017-06-12","cited_by_count":60361},{"id":"1810.04805","title":"BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding","created":"2018-10-11","cited_by_count":40399},{"id":"2005.14165","title":"Language Models are Few-Shot Learners","created":"2020-05-28","cited_by_count":32419},{"id":"2010.11929","title":"An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale","created":"2020-10-22","cited_by_count":30186},{"id":"2203.02155","title":"Training language models to follow instructions with human feedback","created":"2022-03-04","cited_by_count":15971},{"id":"1910.10683","title":"Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer","created":"2019-10-23","cited_by_count":14599},{"id":"1907.11692","title":"RoBERTa: A Robustly Optimized BERT Pretraining Approach","created":"2019-07-26","cited_by_count":13372},{"id":"1708.07747","title":"Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms","created":"2017-08-25","cited_by_count":4349},{"id":"2102.12092","title":"Zero-Shot Text-to-Image Generation","created":"2021-02-24","cited_by_count":4288},{"id":"2111.00396","title":"Efficiently Modeling Long Sequences with Structured State Spaces","created":"2021-10-31","cited_by_count":2465},{"id":"2006.16236","title":"Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention","created":"2020-06-29","cited_by_count":2022},{"id":"2206.04615","title":"Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models","created":"2022-06-09","cited_by_count":1998},{"id":"1901.02860","title":"Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context","created":"2019-01-09","cited_by_count":1850},{"id":"2007.14062","title":"Big Bird: Transformers for Longer Sequences","created":"2020-07-28","cited_by_count":1684},{"id":"2001.04451","title":"Reformer: The Efficient Transformer","created":"2020-01-13","cited_by_count":1668},{"id":"2009.14794","title":"Rethinking Attention with Performers","created":"2020-09-30","cited_by_count":1601},{"id":"2206.14858","title":"Solving Quantitative Reasoning Problems with Language Models","created":"2022-06-29","cited_by_count":1392},{"id":"2112.04426","title":"Improving language models by retrieving from trillions of tokens","created":"2021-12-08","cited_by_count":1293},{"id":"2201.08239","title":"LaMDA: Language Models for Dialog Applications","created":"2022-01-20","cited_by_count":1220},{"id":"1803.02155","title":"Self-Attention with Relative Position Representations","created":"2018-03-06","cited_by_count":1120},{"id":"2108.12409","title":"Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation","created":"2021-08-27","cited_by_count":1103},{"id":"2101.11986","title":"Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet","created":"2021-01-28","cited_by_count":1101},{"id":"2112.06905","title":"GLaM: Efficient Scaling of Language Models with Mixture-of-Experts","created":"2021-12-13","cited_by_count":897},{"id":"2205.06175","title":"A Generalist Agent","created":"2022-05-12","cited_by_count":808},{"id":"2011.04006","title":"Long Range Arena: A Benchmark for Efficient Transformers","created":"2020-11-08","cited_by_count":700},{"id":"1807.03819","title":"Universal Transformers","created":"2018-07-10","cited_by_count":647},{"id":"1907.00235","title":"Enhancing the Locality and Breaking the Memory Bottleneck of Transformer on Time Series Forecasting","created":"2019-06-29","cited_by_count":607},{"id":"2003.05997","title":"Efficient Content-Based Sparse Attention with Routing Transformers","created":"2020-03-12","cited_by_count":471},{"id":"2103.02143","title":"Random Feature Attention","created":"2021-03-03","cited_by_count":354},{"id":"2106.08903","title":"GemNet: Universal Directional Graph Neural Networks for Molecules","created":"2021-06-02","cited_by_count":343},{"id":"2010.09990","title":"The Open Catalyst 2020 (OC20) Dataset and Community Challenges","created":"2020-10-20","cited_by_count":246},{"id":"2103.12731","title":"Scaling Local Self-Attention for Parameter Efficient Visual Backbones","created":"2021-03-23","cited_by_count":220},{"id":"1908.11775","title":"Transformer Dissection: A Unified Understanding of Transformer's Attention via the Lens of Kernel","created":"2019-08-30","cited_by_count":213},{"id":"2202.08791","title":"cosFormer: Rethinking Softmax in Attention","created":"2022-02-17","cited_by_count":213},{"id":"2107.14222","title":"Rethinking and Improving Relative Position Encoding for Vision Transformer","created":"2021-07-29","cited_by_count":207},{"id":"2007.04825","title":"Fast Transformers with Clustered Attention","created":"2020-07-09","cited_by_count":123},{"id":"2102.08602","title":"LambdaNetworks: Modeling Long-Range Interactions Without Attention","created":"2021-02-17","cited_by_count":107},{"id":"2006.04862","title":"$O(n)$ Connections are Expressive Enough: Universal Approximability of Sparse Transformers","created":"2020-06-08","cited_by_count":83},{"id":"2109.04553","title":"Is Attention Better Than Matrix Decomposition?","created":"2021-09-09","cited_by_count":77},{"id":"2210.01765","title":"One Transformer Can Understand Both 2D & 3D Molecular Data","created":"2022-10-04","cited_by_count":77},{"id":"1804.09893","title":"Random Fourier Features for Kernel Ridge Regression: Approximation Bounds and Statistical Guarantees","created":"2018-04-26","cited_by_count":71},{"id":"2203.04810","title":"Benchmarking Graphormer on Large-Scale Molecular Modeling Datasets","created":"2022-03-09","cited_by_count":64},{"id":"2205.13401","title":"Your Transformer May Not be as Powerful as You Expect","created":"2022-05-26","cited_by_count":55},{"id":"2106.12566","title":"Stable, Fast and Accurate: Kernelized Attention with Relative Positional Encoding","created":"2021-06-23","cited_by_count":51},{"id":"2106.08367","title":"What Context Features Can Transformer Language Models Use?","created":"2021-06-15","cited_by_count":50},{"id":"2011.04823","title":"Language Through a Prism: A Spectral Approach for Multiscale Language Representations","created":"2020-11-09","cited_by_count":39},{"id":"2107.07999","title":"From block-Toeplitz matrices to differential equations on graphs: towards a general theory for scalable masked Transformers","created":"2021-07-16","cited_by_count":38},{"id":"2105.08399","title":"Relative Positional Encoding for Transformers with Linear Complexity","created":"2021-05-18","cited_by_count":37},{"id":"2209.10780","title":"Learning Model Predictive Controllers with Real-Time Attention for Real-World Navigation","created":"2022-09-22","cited_by_count":29},{"id":"2110.04367","title":"Hybrid Random Features","created":"2021-10-08","cited_by_count":25},{"id":"2205.15317","title":"Chefs' Random Tables: Non-Trigonometric Random Features","created":"2022-05-30","cited_by_count":21},{"id":"2111.01353","title":"Can Vision Transformers Perform Convolution?","created":"2021-11-02","cited_by_count":18},{"id":"2112.12786","title":"ELSA: Enhanced Local Self-Attention for Vision Transformer","created":"2021-12-23","cited_by_count":18},{"id":"2110.08323","title":"On Learning the Transformer Kernel","created":"2021-10-15","cited_by_count":15},{"id":"2202.06521","title":"Source Code Summarization with Structural Relative Position Guided Transformer","created":"2022-02-14","cited_by_count":14},{"id":"2107.05984","title":"Deep Autoregressive Models with Spectral Attention","created":"2021-07-13","cited_by_count":9},{"id":"2302.01128","title":"Mnemosyne: Learning to Train Transformers with Transformers","created":"2023-02-02","cited_by_count":9},{"id":"1710.01278","title":"Dilated Convolutions for Modeling Long-Distance Genomic Dependencies","created":"2017-10-03","cited_by_count":7},{"id":"2102.13079","title":"Quantization Algorithms for Random Fourier Features","created":"2021-02-25","cited_by_count":7},{"id":"1810.05207","title":"On Kernel Derivative Approximation with Random Fourier Features","created":"2018-10-11","cited_by_count":6},{"id":"2102.07680","title":"Translational Equivariance in Kernelizable Attention","created":"2021-02-15","cited_by_count":5}],"next_cursor":null}