{"list":"other","total":105,"rows":[{"id":"1706.03762","title":"Attention Is All You Need","created":"2017-06-12","cited_by_count":60361},{"id":"1810.04805","title":"BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding","created":"2018-10-11","cited_by_count":40399},{"id":"2005.14165","title":"Language Models are Few-Shot Learners","created":"2020-05-28","cited_by_count":32419},{"id":"1910.10683","title":"Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer","created":"2019-10-23","cited_by_count":14599},{"id":"2201.11903","title":"Chain-of-Thought Prompting Elicits Reasoning in Large Language Models","created":"2022-01-28","cited_by_count":12459},{"id":"2009.03300","title":"Measuring Massive Multitask Language Understanding","created":"2020-09-07","cited_by_count":7963},{"id":"1607.06450","title":"Layer Normalization","created":"2016-07-21","cited_by_count":5669},{"id":"2204.02311","title":"PaLM: Scaling Language Modeling with Pathways","created":"2022-04-05","cited_by_count":5326},{"id":"1803.05457","title":"Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge","created":"2018-03-14","cited_by_count":4886},{"id":"2104.09864","title":"RoFormer: Enhanced Transformer with Rotary Position Embedding","created":"2021-04-20","cited_by_count":4703},{"id":"1910.01108","title":"DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter","created":"2019-10-02","cited_by_count":4307},{"id":"2102.12092","title":"Zero-Shot Text-to-Image Generation","created":"2021-02-24","cited_by_count":4288},{"id":"1902.00751","title":"Parameter-Efficient Transfer Learning for NLP","created":"2019-02-02","cited_by_count":4236},{"id":"2108.07258","title":"On the Opportunities and Risks of Foundation Models","created":"2021-08-16","cited_by_count":4033},{"id":"2101.00190","title":"Prefix-Tuning: Optimizing Continuous Prompts for Generation","created":"2021-01-01","cited_by_count":3915},{"id":"2104.08691","title":"The Power of Scale for Parameter-Efficient Prompt Tuning","created":"2021-04-18","cited_by_count":3795},{"id":"2109.01652","title":"Finetuned Language Models Are Zero-Shot Learners","created":"2021-09-03","cited_by_count":3780},{"id":"2205.01068","title":"OPT: Open Pre-trained Transformer Language Models","created":"2022-05-02","cited_by_count":3512},{"id":"1606.08415","title":"Gaussian Error Linear Units (GELUs)","created":"2016-06-27","cited_by_count":3502},{"id":"2203.15556","title":"Training Compute-Optimal Large Language Models","created":"2022-03-29","cited_by_count":3019},{"id":"2109.07958","title":"TruthfulQA: Measuring How Models Mimic Human Falsehoods","created":"2021-09-08","cited_by_count":2975},{"id":"1705.03551","title":"TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension","created":"2017-05-09","cited_by_count":2955},{"id":"1911.11641","title":"PIQA: Reasoning about Physical Commonsense in Natural Language","created":"2019-11-26","cited_by_count":2883},{"id":"2206.07682","title":"Emergent Abilities of Large Language Models","created":"2022-06-15","cited_by_count":2512},{"id":"2101.00027","title":"The Pile: An 800GB Dataset of Diverse Text for Language Modeling","created":"2020-12-31","cited_by_count":2411},{"id":"2107.13586","title":"Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing","created":"2021-07-28","cited_by_count":2234},{"id":"1809.02789","title":"Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering","created":"2018-09-08","cited_by_count":2164},{"id":"2206.04615","title":"Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models","created":"2022-06-09","cited_by_count":1998},{"id":"1909.08053","title":"Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism","created":"2019-09-17","cited_by_count":1868},{"id":"1901.02860","title":"Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context","created":"2019-01-09","cited_by_count":1850},{"id":"1811.00937","title":"CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge","created":"2018-11-02","cited_by_count":1822},{"id":"1905.00537","title":"SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems","created":"2019-05-02","cited_by_count":1729},{"id":"2104.02057","title":"An Empirical Study of Training Self-Supervised Vision Transformers","created":"2021-04-05","cited_by_count":1616},{"id":"2110.08207","title":"Multitask Prompted Training Enables Zero-Shot Task Generalization","created":"2021-10-15","cited_by_count":1470},{"id":"2002.10957","title":"MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers","created":"2020-02-25","cited_by_count":1463},{"id":"2009.11462","title":"RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models","created":"2020-09-24","cited_by_count":1255},{"id":"1906.02243","title":"Energy and Policy Considerations for Deep Learning in NLP","created":"2019-06-05","cited_by_count":1254},{"id":"2201.08239","title":"LaMDA: Language Models for Dialog Applications","created":"2022-01-20","cited_by_count":1220},{"id":"2112.11446","title":"Scaling Language Models: Methods, Analysis & Insights from Training Gopher","created":"2021-12-08","cited_by_count":1202},{"id":"1909.10351","title":"TinyBERT: Distilling BERT for Natural Language Understanding","created":"2019-09-23","cited_by_count":1167},{"id":"1710.03740","title":"Mixed Precision Training","created":"2017-10-10","cited_by_count":1156},{"id":"2108.12409","title":"Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation","created":"2021-08-27","cited_by_count":1103},{"id":"2103.10360","title":"GLM: General Language Model Pretraining with Autoregressive Blank Infilling","created":"2021-03-18","cited_by_count":1031},{"id":"1606.06031","title":"The LAMBADA dataset: Word prediction requiring a broad discourse context","created":"2016-06-20","cited_by_count":992},{"id":"2205.15868","title":"CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformers","created":"2022-05-29","cited_by_count":985},{"id":"2112.04359","title":"Ethical and social risks of harm from Language Models","created":"2021-12-08","cited_by_count":966},{"id":"2004.09456","title":"StereoSet: Measuring stereotypical bias in pretrained language models","created":"2020-04-20","cited_by_count":924},{"id":"2002.04745","title":"On Layer Normalization in the Transformer Architecture","created":"2020-02-12","cited_by_count":922},{"id":"2208.07339","title":"LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale","created":"2022-08-15","cited_by_count":906},{"id":"1905.10650","title":"Are Sixteen Heads Really Better than One?","created":"2019-05-25","cited_by_count":816},{"id":"2101.02235","title":"Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategies","created":"2021-01-06","cited_by_count":743},{"id":"2002.08910","title":"How Much Knowledge Can You Pack Into the Parameters of a Language Model?","created":"2020-02-10","cited_by_count":719},{"id":"2103.10385","title":"GPT Understands, Too","created":"2021-03-18","cited_by_count":716},{"id":"2204.06745","title":"GPT-NeoX-20B: An Open-Source Autoregressive Language Model","created":"2022-04-14","cited_by_count":713},{"id":"2010.00133","title":"CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models","created":"2020-09-30","cited_by_count":710},{"id":"2006.08218","title":"Self-supervised Learning: Generative or Contrastive","created":"2020-06-15","cited_by_count":694},{"id":"2105.13290","title":"CogView: Mastering Text-to-Image Generation via Transformers","created":"2021-05-26","cited_by_count":694},{"id":"1905.03197","title":"Unified Language Model Pre-training for Natural Language Understanding and Generation","created":"2019-05-08","cited_by_count":670},{"id":"2104.10350","title":"Carbon Emissions and Large Neural Network Training","created":"2021-04-21","cited_by_count":567},{"id":"2201.11990","title":"Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model","created":"2022-01-28","cited_by_count":559},{"id":"2104.08164","title":"Editing Factual Knowledge in Language Models","created":"2021-04-16","cited_by_count":553},{"id":"1707.00600","title":"Zero-Shot Learning -- A Comprehensive Evaluation of the Good, the Bad and the Ugly","created":"2017-07-03","cited_by_count":538},{"id":"1910.09700","title":"Quantifying the Carbon Emissions of Machine Learning","created":"2019-10-21","cited_by_count":521},{"id":"2003.08271","title":"Pre-trained Models for Natural Language Processing: A Survey","created":"2020-03-18","cited_by_count":505},{"id":"1906.04043","title":"GLTR: Statistical Detection and Visualization of Generated Text","created":"2019-06-10","cited_by_count":496},{"id":"2206.06520","title":"Memory-Based Model Editing at Scale","created":"2022-06-13","cited_by_count":447},{"id":"1804.09301","title":"Gender Bias in Coreference Resolution","created":"2018-04-25","cited_by_count":442},{"id":"1909.11556","title":"Reducing Transformer Depth on Demand with Structured Dropout","created":"2019-09-25","cited_by_count":439},{"id":"2110.02861","title":"8-bit Optimizers via Block-wise Quantization","created":"2021-10-06","cited_by_count":403},{"id":"1909.05840","title":"Q-BERT: Hessian Based Ultra Low Precision Quantization of BERT","created":"2019-09-12","cited_by_count":359},{"id":"2103.00453","title":"Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLP","created":"2021-02-28","cited_by_count":337},{"id":"1801.10198","title":"Generating Wikipedia by Summarizing Long Sequences","created":"2018-01-30","cited_by_count":335},{"id":"1912.05511","title":"Measurement and Fairness","created":"2019-12-11","cited_by_count":308},{"id":"1910.06188","title":"Q8BERT: Quantized 8Bit BERT","created":"2019-10-14","cited_by_count":300},{"id":"2106.13219","title":"Towards Understanding and Mitigating Social Biases in Language Models","created":"2021-06-24","cited_by_count":296},{"id":"2202.01279","title":"PromptSource: An Integrated Development Environment and Repository for Natural Language Prompts","created":"2022-02-02","cited_by_count":289},{"id":"2004.05986","title":"CLUE: A Chinese Language Understanding Evaluation Benchmark","created":"2020-04-13","cited_by_count":278},{"id":"1909.03546","title":"Entity, Relation, and Event Extraction with Contextualized Span Representations","created":"2019-09-08","cited_by_count":262},{"id":"2102.01672","title":"The GEM Benchmark: Natural Language Generation, its Evaluation and Metrics","created":"2021-02-02","cited_by_count":239},{"id":"2012.00363","title":"Modifying Memories in Transformer Models","created":"2020-12-01","cited_by_count":224},{"id":"1907.10641","title":"WinoGrande: An Adversarial Winograd Schema Challenge at Scale","created":"2019-07-24","cited_by_count":204},{"id":"2105.04054","title":"Societal Biases in Language Generation: Progress and Challenges","created":"2021-05-10","cited_by_count":202},{"id":"2203.00555","title":"DeepNet: Scaling Transformers to 1,000 Layers","created":"2022-03-01","cited_by_count":178},{"id":"2112.10684","title":"Efficient Large Scale Language Modeling with Mixtures of Experts","created":"2021-12-20","cited_by_count":175},{"id":"2111.10952","title":"ExT5: Towards Extreme Multi-Task Scaling for Transfer Learning","created":"2021-11-22","cited_by_count":161},{"id":"2112.12938","title":"Counterfactual Memorization in Neural Language Models","created":"2021-12-24","cited_by_count":156},{"id":"2006.09503","title":"Memory-Efficient Pipeline-Parallel DNN Training","created":"2020-06-16","cited_by_count":134},{"id":"2104.12369","title":"PanGu-$\\alpha$: Large-scale Autoregressive Pretrained Chinese Language Models with Auto-parallel Computation","created":"2021-04-26","cited_by_count":134},{"id":"2108.12084","title":"Harms of Gender Exclusivity and Challenges in Non-Binary Representation in Language Technologies","created":"2021-08-27","cited_by_count":128},{"id":"1909.03065","title":"\"Going on a vacation\" takes longer than \"Going for a walk\": A Study of Temporal Commonsense Understanding","created":"2019-09-06","cited_by_count":123},{"id":"2004.14900","title":"MLSUM: The Multilingual Summarization Corpus","created":"2020-04-30","cited_by_count":121},{"id":"2207.00747","title":"Rationale-Augmented Ensembles in Language Models","created":"2022-07-02","cited_by_count":111},{"id":"2110.06341","title":"Learning Compact Metrics for MT","created":"2021-10-12","cited_by_count":93},{"id":"2110.09456","title":"NormFormer: Improved Transformer Pretraining with Extra Normalization","created":"2021-10-18","cited_by_count":83},{"id":"2111.13654","title":"Do Language Models Have Beliefs? Methods for Detecting, Updating, and Visualizing Model Beliefs","created":"2021-11-26","cited_by_count":79},{"id":"2203.10705","title":"Compression of Generative Pre-trained Language Models via Quantization","created":"2022-03-21","cited_by_count":79},{"id":"1911.03905","title":"Semantic Noise Matters for Neural Natural Language Generation","created":"2019-11-10","cited_by_count":64},{"id":"2012.11747","title":"RealFormer: Transformer Likes Residual Attention","created":"2020-12-21","cited_by_count":62},{"id":"2103.12407","title":"Detecting Hate Speech with GPT-3","created":"2021-03-23","cited_by_count":56},{"id":"2112.12731","title":"ERNIE 3.0 Titan: Exploring Larger-scale Knowledge Enhanced Pre-training for Language Understanding and Generation","created":"2021-12-23","cited_by_count":46}],"next_cursor":"100"}