{"list":"other","total":65,"rows":[{"id":"1706.03762","title":"Attention Is All You Need","created":"2017-06-12","cited_by_count":60361},{"id":"1810.04805","title":"BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding","created":"2018-10-11","cited_by_count":40399},{"id":"2005.14165","title":"Language Models are Few-Shot Learners","created":"2020-05-28","cited_by_count":32419},{"id":"2307.09288","title":"Llama 2: Open Foundation and Fine-Tuned Chat Models","created":"2023-07-18","cited_by_count":13028},{"id":"2201.11903","title":"Chain-of-Thought Prompting Elicits Reasoning in Large Language Models","created":"2022-01-28","cited_by_count":12459},{"id":"2305.18290","title":"Direct Preference Optimization: Your Language Model is Secretly a Reward Model","created":"2023-05-29","cited_by_count":8065},{"id":"2204.02311","title":"PaLM: Scaling Language Modeling with Pathways","created":"2022-04-05","cited_by_count":5326},{"id":"2205.01068","title":"OPT: Open Pre-trained Transformer Language Models","created":"2022-05-02","cited_by_count":3512},{"id":"1609.07843","title":"Pointer Sentinel Mixture Models","created":"2016-09-26","cited_by_count":3096},{"id":"2001.04451","title":"Reformer: The Efficient Transformer","created":"2020-01-13","cited_by_count":1668},{"id":"1609.08144","title":"Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation","created":"2016-09-26","cited_by_count":1475},{"id":"1811.03962","title":"A Convergence Theory for Deep Learning via Over-Parameterization","created":"2018-11-09","cited_by_count":787},{"id":"2301.10226","title":"A Watermark for Large Language Models","created":"2023-01-24","cited_by_count":725},{"id":"2011.04006","title":"Long Range Arena: A Benchmark for Efficient Transformers","created":"2020-11-08","cited_by_count":700},{"id":"1810.02054","title":"Gradient Descent Provably Optimizes Over-parameterized Neural Networks","created":"2018-10-04","cited_by_count":659},{"id":"2012.12556","title":"A Survey on Visual Transformer","created":"2020-12-23","cited_by_count":654},{"id":"1904.11955","title":"On Exact Computation with an Infinitely Wide Neural Net","created":"2019-04-26","cited_by_count":573},{"id":"1901.08584","title":"Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networks","created":"2019-01-24","cited_by_count":566},{"id":"2306.02561","title":"LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusion","created":"2023-06-05","cited_by_count":564},{"id":"2209.12152","title":"All are Worth Words: A ViT Backbone for Diffusion Models","created":"2022-09-25","cited_by_count":502},{"id":"2305.17333","title":"Fine-Tuning Language Models with Just Forward Passes","created":"2023-05-27","cited_by_count":345},{"id":"2305.14342","title":"Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training","created":"2023-05-23","cited_by_count":265},{"id":"1808.01204","title":"Learning Overparameterized Neural Networks via Stochastic Gradient Descent on Structured Data","created":"2018-08-03","cited_by_count":261},{"id":"2005.08025","title":"IntelliCode Compose: Code Generation Using Transformer","created":"2020-05-16","cited_by_count":227},{"id":"2010.11918","title":"AdapterDrop: On the Efficiency of Adapters in Transformers","created":"2020-10-22","cited_by_count":224},{"id":"1905.13210","title":"Generalization Bounds of Stochastic Gradient Descent for Wide and Deep Neural Networks","created":"2019-05-30","cited_by_count":222},{"id":"2005.00743","title":"Synthesizer: Rethinking Self-Attention in Transformer Models","created":"2020-05-02","cited_by_count":205},{"id":"1902.04674","title":"Towards moderate overparameterization: global convergence guarantees for training shallow neural networks","created":"2019-02-12","cited_by_count":176},{"id":"2206.03126","title":"Signal Propagation in Transformers: Theoretical Perspectives and the Role of Rank Collapse","created":"2022-06-07","cited_by_count":141},{"id":"2306.02896","title":"Representational Strengths and Limitations of Transformers","created":"2023-06-05","cited_by_count":133},{"id":"2110.15343","title":"Scatterbrain: Unifying Sparse and Low-rank Attention Approximation","created":"2021-10-28","cited_by_count":130},{"id":"2301.11798","title":"MedSegDiff-V2: Diffusion based Medical Image Segmentation with Transformer","created":"2023-01-19","cited_by_count":124},{"id":"1906.04688","title":"An Improved Analysis of Training Over-parameterized Deep Neural Networks","created":"2019-06-11","cited_by_count":121},{"id":"1909.12292","title":"Polylogarithmic width suffices for gradient descent to achieve arbitrarily small test error with shallow ReLU networks","created":"2019-09-26","cited_by_count":121},{"id":"2307.15936","title":"A Theory for Emergence of Complex Skills in Language Models","created":"2023-07-29","cited_by_count":113},{"id":"2302.13214","title":"Fast Attention Requires Bounded Entries","created":"2023-02-26","cited_by_count":102},{"id":"2302.10870","title":"On Provable Copyright Protection for Generative Models","created":"2023-02-21","cited_by_count":98},{"id":"2112.02701","title":"Protecting Intellectual Property of Language Generation APIs with Lexical Watermark","created":"2021-12-05","cited_by_count":84},{"id":"1810.12065","title":"On the Convergence Rate of Training Recurrent Neural Networks","created":"2018-10-29","cited_by_count":82},{"id":"1905.10961","title":"Fast Convergence of Natural Gradient Descent for Overparameterized Neural Networks","created":"2019-05-27","cited_by_count":76},{"id":"2209.08773","title":"CATER: Intellectual Property Protection on Text Generation APIs via Conditional Watermarks","created":"2022-09-19","cited_by_count":76},{"id":"2305.18465","title":"Federated Learning of Gboard Language Models with Differential Privacy","created":"2023-05-29","cited_by_count":66},{"id":"2112.07628","title":"Training Multi-Layer Over-Parametrized Neural Network in Subquadratic Time","created":"2021-12-14","cited_by_count":62},{"id":"1906.03593","title":"Quadratic Suffices for Over-parametrization via Matrix Chernoff Bound","created":"2019-06-09","cited_by_count":59},{"id":"2006.11648","title":"Training (Overparametrized) Neural Networks in Near-Linear Time","created":"2020-06-20","cited_by_count":58},{"id":"2304.10411","title":"Attention Scheme Inspired Softmax Regression","created":"2023-04-20","cited_by_count":49},{"id":"2303.16504","title":"An Over-parameterized Exponential Regression","created":"2023-03-29","cited_by_count":45},{"id":"2303.15725","title":"Solving Regularized Exp, Cosh and Sinh Regression Problems","created":"2023-03-28","cited_by_count":44},{"id":"2304.13276","title":"The Closeness of In-Context Learning and Weight Shifting for Softmax Regression","created":"2023-04-26","cited_by_count":41},{"id":"1905.11675","title":"Gram-Gauss-Newton Method: Learning Overparameterized Neural Networks for Regression Problems","created":"2019-05-28","cited_by_count":40},{"id":"2304.02207","title":"Algorithm and Hardness for Dynamic Attention Maintenance in Large Language Models","created":"2023-04-05","cited_by_count":37},{"id":"2002.06668","title":"Over-parameterized Adversarial Training: An Analysis Overcoming the Curse of Dimensionality","created":"2020-02-16","cited_by_count":36},{"id":"2306.04933","title":"InfoPrompt: Information-Theoretic Soft Prompt Tuning for Natural Language Understanding","created":"2023-06-08","cited_by_count":30},{"id":"2211.14227","title":"Bypass Exponential Time Preprocessing: Fast Neural Network Training via Weight-Data Correlation Preprocessing","created":"2022-11-25","cited_by_count":29},{"id":"2009.09829","title":"Generalized Leverage Score Sampling for Neural Networks","created":"2020-09-21","cited_by_count":26},{"id":"2305.04701","title":"Differentially Private Attention Computation","created":"2023-05-08","cited_by_count":25},{"id":"2308.10502","title":"GradientCoin: A Peer-to-Peer Decentralized Large Language Models","created":"2023-08-21","cited_by_count":23},{"id":"2306.10015","title":"Just One Byte (per gradient): A Note on Low-Bandwidth Decentralized Language Model Finetuning Using Shared Randomness","created":"2023-06-16","cited_by_count":18},{"id":"2307.08045","title":"Beyond Classical Attention: Quantum Attention for Scalable Computation","created":"2023-07-16","cited_by_count":18},{"id":"2307.01189","title":"Trainable Transformer in Transformer","created":"2023-07-03","cited_by_count":17},{"id":"2307.08352","title":"Zero-th Order Algorithm for Softmax Attention Optimization","created":"2023-07-17","cited_by_count":16},{"id":"2206.12802","title":"Bounding the Width of Neural Networks via Coupled Initialization -- A Worst Case Analysis","created":"2022-06-26","cited_by_count":15},{"id":"2307.06565","title":"Efficient SGD Neural Network Training via Sublinear Activated Neuron Identification","created":"2023-07-13","cited_by_count":11},{"id":"2212.13771","title":"Exploring Vision Transformers as Diffusion Learners","created":"2022-12-28","cited_by_count":10},{"id":"2307.16382","title":"Does fine-tuning GPT-3 with the OpenAI API leak personally-identifiable information?","created":"2023-07-31","cited_by_count":10}],"next_cursor":null}