{"list":"other","total":71,"rows":[{"id":"1706.03762","title":"Attention Is All You Need","created":"2017-06-12","cited_by_count":60361},{"id":"1810.04805","title":"BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding","created":"2018-10-11","cited_by_count":40399},{"id":"2005.14165","title":"Language Models are Few-Shot Learners","created":"2020-05-28","cited_by_count":32419},{"id":"1910.10683","title":"Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer","created":"2019-10-23","cited_by_count":14599},{"id":"2106.09685","title":"LoRA: Low-Rank Adaptation of Large Language Models","created":"2021-06-17","cited_by_count":14380},{"id":"2302.13971","title":"LLaMA: Open and Efficient Foundation Language Models","created":"2023-02-27","cited_by_count":14132},{"id":"2307.09288","title":"Llama 2: Open Foundation and Fine-Tuned Chat Models","created":"2023-07-18","cited_by_count":13028},{"id":"2001.08361","title":"Scaling Laws for Neural Language Models","created":"2020-01-23","cited_by_count":6586},{"id":"2205.11487","title":"Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding","created":"2022-05-23","cited_by_count":6355},{"id":"1607.06450","title":"Layer Normalization","created":"2016-07-21","cited_by_count":5669},{"id":"2204.02311","title":"PaLM: Scaling Language Modeling with Pathways","created":"2022-04-05","cited_by_count":5326},{"id":"2204.14198","title":"Flamingo: a Visual Language Model for Few-Shot Learning","created":"2022-04-29","cited_by_count":4386},{"id":"2204.05862","title":"Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback","created":"2022-04-12","cited_by_count":3720},{"id":"2205.14135","title":"FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness","created":"2022-05-27","cited_by_count":3673},{"id":"1609.07843","title":"Pointer Sentinel Mixture Models","created":"2016-09-26","cited_by_count":3096},{"id":"2212.08073","title":"Constitutional AI: Harmlessness from AI Feedback","created":"2022-12-15","cited_by_count":3050},{"id":"2203.15556","title":"Training Compute-Optimal Large Language Models","created":"2022-03-29","cited_by_count":3019},{"id":"1508.07909","title":"Neural Machine Translation of Rare Words with Subword Units","created":"2015-08-31","cited_by_count":2918},{"id":"2303.18223","title":"A Survey of Large Language Models","created":"2023-03-31","cited_by_count":2576},{"id":"2101.00027","title":"The Pile: An 800GB Dataset of Diverse Text for Language Modeling","created":"2020-12-31","cited_by_count":2411},{"id":"2107.13586","title":"Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing","created":"2021-07-28","cited_by_count":2234},{"id":"2303.03378","title":"PaLM-E: An Embodied Multimodal Language Model","created":"2023-03-06","cited_by_count":2071},{"id":"2006.16236","title":"Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention","created":"2020-06-29","cited_by_count":2022},{"id":"1808.06226","title":"SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing","created":"2018-08-19","cited_by_count":1989},{"id":"2211.05100","title":"BLOOM: A 176B-Parameter Open-Access Multilingual Language Model","created":"2022-11-09","cited_by_count":1984},{"id":"1909.08053","title":"Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism","created":"2019-09-17","cited_by_count":1868},{"id":"2304.01373","title":"Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling","created":"2023-04-03","cited_by_count":1786},{"id":"2108.12409","title":"Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation","created":"2021-08-27","cited_by_count":1103},{"id":"1607.01759","title":"Bag of Tricks for Efficient Text Classification","created":"2016-07-06","cited_by_count":876},{"id":"2305.13048","title":"RWKV: Reinventing RNNs for the Transformer Era","created":"2023-05-22","cited_by_count":827},{"id":"1911.05507","title":"Compressive Transformers for Long-Range Sequence Modelling","created":"2019-11-13","cited_by_count":726},{"id":"2308.10792","title":"Instruction Tuning for Large Language Models: A Survey","created":"2023-08-21","cited_by_count":623},{"id":"2102.11174","title":"Linear Transformers Are Secretly Fast Weight Programmers","created":"2021-02-22","cited_by_count":502},{"id":"2212.14052","title":"Hungry Hungry Hippos: Towards Language Modeling with State Space Models","created":"2022-12-28","cited_by_count":499},{"id":"1612.03651","title":"FastText.zip: Compressing text classification models","created":"2016-12-12","cited_by_count":335},{"id":"1711.00489","title":"Don't Decay the Learning Rate, Increase the Batch Size","created":"2017-11-01","cited_by_count":320},{"id":"1812.06162","title":"An Empirical Model of Large-Batch Training","created":"2018-12-14","cited_by_count":281},{"id":"1711.10337","title":"Are GANs Created Equal? A Large-Scale Study","created":"2017-11-28","cited_by_count":267},{"id":"1505.00387","title":"Highway Networks","created":"2015-05-03","cited_by_count":260},{"id":"2207.04901","title":"Exploring Length Generalization in Large Language Models","created":"2022-07-11","cited_by_count":206},{"id":"1811.03600","title":"Measuring the Effects of Data Parallelism on Neural Network Training","created":"2018-11-08","cited_by_count":193},{"id":"1907.06902","title":"Are We Really Making Much Progress? A Worrying Analysis of Recent Neural Recommendation Approaches","created":"2019-07-16","cited_by_count":185},{"id":"2009.06489","title":"The Hardware Lottery","created":"2020-09-14","cited_by_count":160},{"id":"2308.12284","title":"D4: Improving LLM Pretraining via Document De-Duplication and Diversification","created":"2023-08-23","cited_by_count":150},{"id":"2303.03915","title":"The BigScience ROOTS Corpus: A 1.6TB Composite Multilingual Dataset","created":"2023-03-07","cited_by_count":148},{"id":"2304.13731","title":"Text-to-Audio Generation using Instruction-Tuned LLM and Latent Diffusion Model","created":"2023-04-24","cited_by_count":145},{"id":"2109.12948","title":"Understanding and Overcoming the Challenges of Efficient Transformer Quantization","created":"2021-09-27","cited_by_count":141},{"id":"1808.04444","title":"Character-Level Language Modeling with Deeper Self-Attention","created":"2018-08-09","cited_by_count":138},{"id":"2307.03170","title":"Focused Transformer: Contrastive Training for Context Scaling","created":"2023-07-06","cited_by_count":133},{"id":"1901.03429","title":"On the Turing Completeness of Modern Neural Network Architectures","created":"2019-01-10","cited_by_count":129},{"id":"1907.04164","title":"Which Algorithmic Choices Matter at Which Batch Sizes? Insights From a Noisy Quadratic Model","created":"2019-07-09","cited_by_count":117},{"id":"2203.07852","title":"Block-Recurrent Transformers","created":"2022-03-11","cited_by_count":110},{"id":"2304.03208","title":"Cerebras-GPT: Open Compute-Optimal Language Models Trained on the Cerebras Wafer-Scale Cluster","created":"2023-04-06","cited_by_count":109},{"id":"2207.10342","title":"Language Model Cascades","created":"2022-07-21","cited_by_count":107},{"id":"2212.04458","title":"General-Purpose In-Context Learning by Meta-Learning Transformers","created":"2022-12-08","cited_by_count":105},{"id":"1910.05446","title":"On Empirical Comparisons of Optimizers for Deep Learning","created":"2019-10-11","cited_by_count":103},{"id":"2207.10551","title":"Scaling Laws vs Model Architectures: How does Inductive Bias Influence Scaling?","created":"2022-07-21","cited_by_count":103},{"id":"1611.01576","title":"Quasi-Recurrent Neural Networks","created":"2016-11-05","cited_by_count":87},{"id":"2102.11972","title":"Do Transformer Modifications Transfer Across Implementations and Applications?","created":"2021-02-23","cited_by_count":87},{"id":"2212.14034","title":"Cramming: Training a Language Model on a Single GPU in One Day","created":"2022-12-28","cited_by_count":81},{"id":"2110.11403","title":"SCENIC: A JAX Library for Computer Vision Research and Beyond","created":"2021-10-18","cited_by_count":57},{"id":"1709.07432","title":"Dynamic Evaluation of Neural Sequence Models","created":"2017-09-21","cited_by_count":55},{"id":"2205.13401","title":"Your Transformer May Not be as Powerful as You Expect","created":"2022-05-26","cited_by_count":55},{"id":"2012.14905","title":"Meta Learning Backpropagation And Improving It","created":"2020-12-29","cited_by_count":49},{"id":"2302.14502","title":"A Survey on Long Text Modeling with Transformers","created":"2023-02-28","cited_by_count":47},{"id":"2202.05780","title":"A Modern Self-Referential Weight Matrix That Learns to Modify Itself","created":"2022-02-11","cited_by_count":41},{"id":"2307.06440","title":"No Train No Gain: Revisiting Efficient Training Algorithms For Transformer-based Language Models","created":"2023-07-12","cited_by_count":41},{"id":"2307.07982","title":"A Survey of Techniques for Optimizing Transformer Inference","created":"2023-07-16","cited_by_count":35},{"id":"2307.14995","title":"TransNormerLLM: A Faster and Better Large Language Model with Improved TransNormer","created":"2023-07-27","cited_by_count":34},{"id":"2305.05364","title":"Large Language Model Programs","created":"2023-05-09","cited_by_count":11},{"id":"2307.01236","title":"Rockmate: an Efficient, Fast, Automatic and Generic Tool for Re-materialization in PyTorch","created":"2023-07-03","cited_by_count":4}],"next_cursor":null}