{"list":"other","total":75,"rows":[{"id":"2005.14165","title":"Language Models are Few-Shot Learners","created":"2020-05-28","cited_by_count":32419},{"id":"2112.10752","title":"High-Resolution Image Synthesis with Latent Diffusion Models","created":"2021-12-20","cited_by_count":18863},{"id":"1910.10683","title":"Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer","created":"2019-10-23","cited_by_count":14599},{"id":"2107.03374","title":"Evaluating Large Language Models Trained on Code","created":"2021-07-07","cited_by_count":8320},{"id":"2001.08361","title":"Scaling Laws for Neural Language Models","created":"2020-01-23","cited_by_count":6586},{"id":"2204.06125","title":"Hierarchical Text-Conditional Image Generation with CLIP Latents","created":"2022-04-13","cited_by_count":6585},{"id":"2204.02311","title":"PaLM: Scaling Language Modeling with Pathways","created":"2022-04-05","cited_by_count":5326},{"id":"2104.09864","title":"RoFormer: Enhanced Transformer with Rotary Position Embedding","created":"2021-04-20","cited_by_count":4703},{"id":"1911.02116","title":"Unsupervised Cross-lingual Representation Learning at Scale","created":"2019-11-05","cited_by_count":3935},{"id":"2205.14135","title":"FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness","created":"2022-05-27","cited_by_count":3673},{"id":"2205.01068","title":"OPT: Open Pre-trained Transformer Language Models","created":"2022-05-02","cited_by_count":3512},{"id":"2203.15556","title":"Training Compute-Optimal Large Language Models","created":"2022-03-29","cited_by_count":3019},{"id":"2101.00027","title":"The Pile: An 800GB Dataset of Diverse Text for Language Modeling","created":"2020-12-31","cited_by_count":2411},{"id":"1910.03771","title":"HuggingFace's Transformers: State-of-the-art Natural Language Processing","created":"2019-10-09","cited_by_count":2218},{"id":"2012.07805","title":"Extracting Training Data from Large Language Models","created":"2020-12-14","cited_by_count":2189},{"id":"1909.08053","title":"Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism","created":"2019-09-17","cited_by_count":1868},{"id":"1706.02677","title":"Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour","created":"2017-06-08","cited_by_count":1492},{"id":"2110.08207","title":"Multitask Prompted Training Enables Zero-Shot Task Generalization","created":"2021-10-15","cited_by_count":1470},{"id":"2212.10511","title":"When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories","created":"2022-12-20","cited_by_count":1128},{"id":"1606.06031","title":"The LAMBADA dataset: Word prediction requiring a broad discourse context","created":"2016-06-20","cited_by_count":992},{"id":"1802.08232","title":"The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks","created":"2018-02-22","cited_by_count":864},{"id":"2210.02414","title":"GLM-130B: An Open Bilingual Pre-trained Model","created":"2022-10-05","cited_by_count":843},{"id":"2202.07646","title":"Quantifying Memorization Across Neural Language Models","created":"2022-02-15","cited_by_count":763},{"id":"2107.06499","title":"Deduplicating Training Data Makes Language Models Better","created":"2021-07-14","cited_by_count":714},{"id":"2204.06745","title":"GPT-NeoX-20B: An Open-Source Autoregressive Language Model","created":"2022-04-14","cited_by_count":713},{"id":"2010.00133","title":"CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models","created":"2020-09-30","cited_by_count":710},{"id":"1804.06876","title":"Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods","created":"2018-04-18","cited_by_count":659},{"id":"2303.08112","title":"Eliciting Latent Predictions from Transformers with the Tuned Lens","created":"2023-03-14","cited_by_count":623},{"id":"2201.11990","title":"Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model","created":"2022-01-28","cited_by_count":559},{"id":"2010.14701","title":"Scaling Laws for Autoregressive Generative Modeling","created":"2020-10-28","cited_by_count":522},{"id":"2204.05999","title":"InCoder: A Generative Model for Code Infilling and Synthesis","created":"2022-04-12","cited_by_count":498},{"id":"2011.03395","title":"Underspecification Presents Challenges for Credibility in Modern Machine Learning","created":"2020-11-06","cited_by_count":496},{"id":"2104.08758","title":"Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpus","created":"2021-04-18","cited_by_count":492},{"id":"2211.08411","title":"Large Language Models Struggle to Learn Long-Tail Knowledge","created":"2022-11-15","cited_by_count":458},{"id":"1707.09457","title":"Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints","created":"2017-07-29","cited_by_count":420},{"id":"2202.13169","title":"A Systematic Evaluation of Large Language Models of Code","created":"2022-02-26","cited_by_count":363},{"id":"2103.07853","title":"Membership Inference Attacks on Machine Learning: A Survey","created":"2021-03-14","cited_by_count":321},{"id":"2204.08583","title":"VQGAN-CLIP: Open Domain Image Generation and Editing with Natural Language Guidance","created":"2022-04-18","cited_by_count":285},{"id":"1812.06162","title":"An Empirical Model of Large-Batch Training","created":"2018-12-14","cited_by_count":281},{"id":"2102.01293","title":"Scaling Laws for Transfer","created":"2021-02-02","cited_by_count":272},{"id":"2103.12028","title":"Quality at a Glance: An Audit of Web-Crawled Multilingual Datasets","created":"2021-03-22","cited_by_count":243},{"id":"2205.10770","title":"Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Models","created":"2022-05-22","cited_by_count":220},{"id":"1904.03035","title":"Identifying and Reducing Gender Bias in Word-Level Language Models","created":"2019-04-05","cited_by_count":214},{"id":"2205.12628","title":"Are Large Pre-Trained Language Models Leaking Your Personal Information?","created":"2022-05-25","cited_by_count":200},{"id":"2106.15590","title":"The Values Encoded in Machine Learning Research","created":"2021-06-29","cited_by_count":172},{"id":"2205.10487","title":"Scaling Laws and Interpretability of Learning from Repeated Data","created":"2022-05-21","cited_by_count":152},{"id":"2303.03915","title":"The BigScience ROOTS Corpus: A 1.6TB Composite Multilingual Dataset","created":"2023-03-07","cited_by_count":148},{"id":"2205.06266","title":"Lifting the Curse of Multilinguality by Pre-training Modular Transformers","created":"2022-05-12","cited_by_count":146},{"id":"2202.07206","title":"Impact of Pretraining Term Frequencies on Few-Shot Reasoning","created":"2022-02-15","cited_by_count":142},{"id":"2111.09259","title":"Acquisition of Chess Knowledge in AlphaZero","created":"2021-11-17","cited_by_count":123},{"id":"2207.00099","title":"Measuring Forgetting of Memorized Training Examples","created":"2022-06-30","cited_by_count":118},{"id":"1907.04164","title":"Which Algorithmic Choices Matter at Which Batch Sizes? Insights From a Noisy Quadratic Model","created":"2019-07-09","cited_by_count":117},{"id":"2109.07740","title":"Scaling Laws for Neural Machine Translation","created":"2021-09-16","cited_by_count":111},{"id":"2106.16163","title":"The MultiBERTs: BERT Reproductions for Robustness Analysis","created":"2021-06-30","cited_by_count":92},{"id":"2210.15424","title":"What Language Model to Train if You Have One Million GPU Hours?","created":"2022-10-27","cited_by_count":92},{"id":"1810.03611","title":"Understanding the Origins of Bias in Word Embeddings","created":"2018-10-08","cited_by_count":91},{"id":"2204.13509","title":"On the Effect of Pretraining Corpora on In-context Learning by a Large-scale Language Model","created":"2022-04-28","cited_by_count":81},{"id":"2212.09803","title":"Training Trajectories of Language Models Across Scales","created":"2022-12-19","cited_by_count":81},{"id":"2010.14571","title":"Language ID in the Wild: Unexpected Challenges on the Path to a Thousand-Language Web Text Corpus","created":"2020-10-27","cited_by_count":77},{"id":"2201.11706","title":"A Systematic Study of Bias Amplification","created":"2022-01-27","cited_by_count":65},{"id":"2109.03858","title":"Collecting a Large-Scale Gender Bias Dataset for Coreference Resolution and Machine Translation","created":"2021-09-08","cited_by_count":62},{"id":"2207.14251","title":"Measuring Causal Effects of Data Statistics on Language Model's `Factual' Predictions","created":"2022-07-28","cited_by_count":57},{"id":"2004.10802","title":"A Neural Scaling Law from the Dimension of the Data Manifold","created":"2020-04-22","cited_by_count":52},{"id":"2201.07311","title":"Datasheet for the Pile","created":"2022-01-13","cited_by_count":43},{"id":"2203.15395","title":"Quantifying Societal Bias Amplification in Image Captioning","created":"2022-03-29","cited_by_count":37},{"id":"2206.03216","title":"Data Governance in the Age of Large-Scale Data-Driven Language Technology","created":"2022-05-04","cited_by_count":33},{"id":"2201.00971","title":"Submix: Practical Private Prediction for Large-Scale Language Models","created":"2022-01-04","cited_by_count":31},{"id":"2109.10052","title":"Stepmothers are mean and academics are pretentious: What do pretrained language models learn about you?","created":"2021-09-21","cited_by_count":16},{"id":"2207.10245","title":"The Birth of Bias: A case study on the evolution of gender bias in an English language model","created":"2022-07-21","cited_by_count":16},{"id":"2108.11018","title":"A Scaling Law for Synthetic-to-Real Transfer: How Much Is Your Pre-training Effective?","created":"2021-08-25","cited_by_count":15},{"id":"2210.06413","title":"EleutherAI: Going Beyond \"Open Science\" to \"Science in the Open\"","created":"2022-10-12","cited_by_count":13},{"id":"2012.01685","title":"Cross-Loss Influence Functions to Explain Deep Network Representations","created":"2020-12-03","cited_by_count":12},{"id":"2201.10066","title":"Documenting Geographically and Contextually Diverse Data Sources: The BigScience Catalogue of Language Data and Resources","created":"2022-01-25","cited_by_count":11},{"id":"2211.13709","title":"Undesirable Biases in NLP: Addressing Challenges of Measurement","created":"2022-11-24","cited_by_count":4},{"id":"2209.03661","title":"Efficient Gender Debiasing of Pre-trained Indic Language Models","created":"2022-09-08","cited_by_count":3}],"next_cursor":null}