{"list":"other","total":73,"rows":[{"id":"1810.04805","title":"BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding","created":"2018-10-11","cited_by_count":40399},{"id":"2005.14165","title":"Language Models are Few-Shot Learners","created":"2020-05-28","cited_by_count":32419},{"id":"1910.10683","title":"Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer","created":"2019-10-23","cited_by_count":14599},{"id":"1907.11692","title":"RoBERTa: A Robustly Optimized BERT Pretraining Approach","created":"2019-07-26","cited_by_count":13372},{"id":"2307.09288","title":"Llama 2: Open Foundation and Fine-Tuned Chat Models","created":"2023-07-18","cited_by_count":13028},{"id":"2001.08361","title":"Scaling Laws for Neural Language Models","created":"2020-01-23","cited_by_count":6586},{"id":"1804.07461","title":"GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding","created":"2018-04-20","cited_by_count":4899},{"id":"2205.01068","title":"OPT: Open Pre-trained Transformer Language Models","created":"2022-05-02","cited_by_count":3512},{"id":"1704.05426","title":"A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference","created":"2017-04-18","cited_by_count":2682},{"id":"2006.03654","title":"DeBERTa: Decoding-enhanced BERT with Disentangled Attention","created":"2020-06-05","cited_by_count":2181},{"id":"2211.05100","title":"BLOOM: A 176B-Parameter Open-Access Multilingual Language Model","created":"2022-11-09","cited_by_count":1984},{"id":"1508.05326","title":"A large annotated corpus for learning natural language inference","created":"2015-08-21","cited_by_count":1847},{"id":"2002.05202","title":"GLU Variants Improve Transformer","created":"2020-02-12","cited_by_count":1804},{"id":"1905.00537","title":"SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems","created":"2019-05-02","cited_by_count":1729},{"id":"1910.14599","title":"Adversarial NLI: A New Benchmark for Natural Language Understanding","created":"2019-10-31","cited_by_count":807},{"id":"1912.00582","title":"BLiMP: The Benchmark of Linguistic Minimal Pairs for English","created":"2019-12-02","cited_by_count":564},{"id":"2104.08164","title":"Editing Factual Knowledge in Language Models","created":"2021-04-16","cited_by_count":553},{"id":"2104.14337","title":"Dynabench: Rethinking Benchmarking in NLP","created":"2021-04-07","cited_by_count":459},{"id":"1808.09031","title":"Targeted Syntactic Evaluation of Language Models","created":"2018-08-27","cited_by_count":242},{"id":"2012.15761","title":"Learning from the Worst: Dynamically Generated Datasets to Improve Online Hate Detection","created":"2020-12-31","cited_by_count":221},{"id":"2306.09479","title":"Inverse Scaling: When Bigger Isn't Better","created":"2023-06-15","cited_by_count":207},{"id":"2208.03306","title":"Branch-Train-Merge: Embarrassingly Parallel Training of Expert Language Models","created":"2022-08-05","cited_by_count":191},{"id":"1910.05895","title":"Transformers without Tears: Improving the Normalization of Self-Attention","created":"2019-10-14","cited_by_count":153},{"id":"2202.08005","title":"Should You Mask 15% in Masked Language Modeling?","created":"2022-02-16","cited_by_count":118},{"id":"2005.00955","title":"How Can We Accelerate Progress Towards Human-like Linguistic Generalization?","created":"2020-05-03","cited_by_count":114},{"id":"1511.02301","title":"The Goldilocks Principle: Reading Children's Books with Explicit Memory Representations","created":"2015-11-07","cited_by_count":108},{"id":"2011.04946","title":"When Do You Need Billions of Words of Pretraining Data?","created":"2020-11-10","cited_by_count":104},{"id":"2208.07998","title":"What Artificial Neural Networks Can Tell Us About Human Language Acquisition","created":"2022-08-17","cited_by_count":99},{"id":"2207.10062","title":"DataPerf: Benchmarks for Data-Centric AI Development","created":"2022-07-20","cited_by_count":90},{"id":"2110.09456","title":"NormFormer: Improved Transformer Pretraining with Extra Normalization","created":"2021-10-18","cited_by_count":83},{"id":"2104.07705","title":"How to Train BERT with an Academic Budget","created":"2021-04-15","cited_by_count":80},{"id":"2104.08678","title":"Improving Question Answering Model Robustness with Synthetic Adversarial Data Generation","created":"2021-04-18","cited_by_count":79},{"id":"2012.15832","title":"Shortformer: Better Language Modeling using Shorter Inputs","created":"2020-12-31","cited_by_count":76},{"id":"2304.08442","title":"The MiniPile Challenge for Data-Efficient Language Models","created":"2023-04-17","cited_by_count":75},{"id":"2001.03632","title":"Does syntax need to grow on trees? Sources of hierarchical inductive bias in sequence-to-sequence networks","created":"2020-01-10","cited_by_count":72},{"id":"1607.08723","title":"Cognitive Science in the era of Artificial Intelligence: A roadmap for reverse-engineering the infant language-learner","created":"2016-07-29","cited_by_count":68},{"id":"1812.08092","title":"A standardized Project Gutenberg corpus for statistical analysis of natural language and quantitative linguistics","created":"2018-12-19","cited_by_count":64},{"id":"2012.15349","title":"DynaSent: A Dynamic Benchmark for Sentiment Analysis","created":"2020-12-30","cited_by_count":63},{"id":"2301.11796","title":"Call for Papers -- The BabyLM Challenge: Sample-efficient pretraining on a developmentally plausible corpus","created":"2023-01-27","cited_by_count":62},{"id":"2106.10207","title":"Distributed Deep Learning in Open Collaborations","created":"2021-06-18","cited_by_count":58},{"id":"2203.13112","title":"minicons: Enabling Flexible Behavioral and Representational Analyses of Transformer Language Models","created":"2022-03-24","cited_by_count":58},{"id":"2106.02280","title":"Human-Adversarial Visual Question Answering","created":"2021-06-04","cited_by_count":57},{"id":"2308.02019","title":"Baby Llama: knowledge distillation from an ensemble of teachers trained on a small dataset with no performance penalty","created":"2023-08-03","cited_by_count":57},{"id":"2212.01378","title":"ColD Fusion: Collaborative Descent for Distributed Multitask Finetuning","created":"2022-12-02","cited_by_count":52},{"id":"2108.05921","title":"Hatemoji: A Test Suite and Adversarially-Generated Dataset for Benchmarking and Detecting Emoji-based Hate","created":"2021-08-12","cited_by_count":50},{"id":"2106.08694","title":"On the proper role of linguistically-oriented deep net analysis in linguistic theorizing","created":"2021-06-16","cited_by_count":43},{"id":"2303.09859","title":"Trained on 100 million words and still in shape: BERT meets British National Corpus","created":"2023-03-17","cited_by_count":39},{"id":"2012.00857","title":"StructFormer: Joint Unsupervised Induction of Dependency and Constituency Structure from Masked Language Modeling","created":"2020-12-01","cited_by_count":33},{"id":"1905.10425","title":"Human vs. Muppet: A Conservative Estimate of Human Performance on the GLUE Benchmark","created":"2019-05-24","cited_by_count":32},{"id":"2203.09397","title":"Coloring the Blank Slate: Pre-training Imparts a Hierarchical Inductive Bias to Sequence-to-sequence Models","created":"2022-03-17","cited_by_count":28},{"id":"1909.00111","title":"Quantity doesn't buy quality syntax with neural language models","created":"2019-08-31","cited_by_count":25},{"id":"2211.00107","title":"Where to start? Analyzing the potential value of intermediate models","created":"2022-10-31","cited_by_count":23},{"id":"2305.19905","title":"How to Plant Trees in Language Models: Data and Architectural Effects on the Emergence of Syntactic Inductive Biases","created":"2023-05-31","cited_by_count":23},{"id":"2311.02265","title":"Not all layers are equally as important: Every Layer Counts BERT","created":"2023-11-03","cited_by_count":21},{"id":"2311.08886","title":"CLIMB: Curriculum Learning for Infant-inspired Model Building","created":"2023-11-15","cited_by_count":20},{"id":"2306.04529","title":"Git-Theta: A Git Extension for Collaborative Development of Machine Learning Models","created":"2023-06-07","cited_by_count":15},{"id":"1609.03207","title":"Multiplex lexical networks reveal patterns in early word acquisition in children","created":"2016-09-11","cited_by_count":11},{"id":"1809.04179","title":"What can linguistics and deep learning contribute to each other?","created":"2018-09-11","cited_by_count":10},{"id":"2010.04637","title":"Recurrent babbling: evaluating the acquisition of grammar from limited input data","created":"2020-10-09","cited_by_count":9},{"id":"2108.04324","title":"FairyTailor: A Multimodal Generative Framework for Storytelling","created":"2021-07-13","cited_by_count":9},{"id":"2308.01684","title":"Baby's CoThought: Leveraging Large Language Models for Enhanced Reasoning in Compact Models","created":"2023-08-03","cited_by_count":9},{"id":"2310.16681","title":"BabyStories: Can Reinforcement Learning Teach Baby Language Models to Write Better Stories?","created":"2023-10-25","cited_by_count":7},{"id":"2311.01955","title":"Too Much Information: Keeping Training Simple for BabyLMs","created":"2023-11-03","cited_by_count":7},{"id":"2311.00128","title":"On the effect of curriculum learning with developmental data for grammar acquisition","created":"2023-10-31","cited_by_count":5},{"id":"2311.04666","title":"Pre-training LLMs using human-like development data corpus","created":"2023-11-08","cited_by_count":5},{"id":"2311.18761","title":"Can training neural language models on a curriculum with developmentally plausible data improve alignment with human reading behavior?","created":"2023-11-30","cited_by_count":5},{"id":"2310.11282","title":"ChapGTP, ILLC's Attempt at Raising a BabyLM: Improving Data Efficiency by Automatic Task Formation","created":"2023-10-17","cited_by_count":3},{"id":"2310.20589","title":"Increasing The Performance of Cognitively Inspired Data-Efficient Language Models via Implicit Structure Building","created":"2023-10-31","cited_by_count":3},{"id":"2402.17936","title":"Acquiring Linguistic Knowledge from Multimodal Input","created":"2024-02-27","cited_by_count":3},{"id":"2312.02931","title":"WhisBERT: Multimodal Text-Audio Language Modeling on 100M Words","created":"2023-12-05","cited_by_count":2},{"id":"2308.16336","title":"ToddlerBERTa: Exploiting BabyBERTa for Grammar Learning and Language Understanding","created":"2023-08-30","cited_by_count":1},{"id":"2310.17591","title":"Lil-Bevo: Explorations of Strategies for Training Language Models in More Humanlike Ways","created":"2023-10-26","cited_by_count":1},{"id":"2311.03216","title":"Mini Minds: Exploring Bebeshka and Zlata Baby Models","created":"2023-11-06","cited_by_count":1}],"next_cursor":null}