{"list":"other","total":60,"rows":[{"id":"2005.14165","title":"Language Models are Few-Shot Learners","created":"2020-05-28","cited_by_count":32419},{"id":"1910.10683","title":"Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer","created":"2019-10-23","cited_by_count":14599},{"id":"2307.09288","title":"Llama 2: Open Foundation and Fine-Tuned Chat Models","created":"2023-07-18","cited_by_count":13028},{"id":"2110.14168","title":"Training Verifiers to Solve Math Word Problems","created":"2021-10-27","cited_by_count":10109},{"id":"2009.03300","title":"Measuring Massive Multitask Language Understanding","created":"2020-09-07","cited_by_count":7963},{"id":"1804.07461","title":"GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding","created":"2018-04-20","cited_by_count":4899},{"id":"1803.05457","title":"Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge","created":"2018-03-14","cited_by_count":4886},{"id":"1905.07830","title":"HellaSwag: Can a Machine Really Finish Your Sentence?","created":"2019-05-19","cited_by_count":4100},{"id":"1609.07843","title":"Pointer Sentinel Mixture Models","created":"2016-09-26","cited_by_count":3096},{"id":"2203.15556","title":"Training Compute-Optimal Large Language Models","created":"2022-03-29","cited_by_count":3019},{"id":"1911.11641","title":"PIQA: Reasoning about Physical Commonsense in Natural Language","created":"2019-11-26","cited_by_count":2883},{"id":"2101.00027","title":"The Pile: An 800GB Dataset of Diverse Text for Language Modeling","created":"2020-12-31","cited_by_count":2411},{"id":"1809.02789","title":"Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering","created":"2018-09-08","cited_by_count":2164},{"id":"1811.00937","title":"CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge","created":"2018-11-02","cited_by_count":1822},{"id":"1703.04730","title":"Understanding Black-box Predictions via Influence Functions","created":"2017-03-14","cited_by_count":1741},{"id":"1911.11763","title":"SuperGlue: Learning Feature Matching with Graph Neural Networks","created":"2019-11-26","cited_by_count":1443},{"id":"2112.11446","title":"Scaling Language Models: Methods, Analysis & Insights from Training Gopher","created":"2021-12-08","cited_by_count":1202},{"id":"1606.06031","title":"The LAMBADA dataset: Word prediction requiring a broad discourse context","created":"2016-06-20","cited_by_count":992},{"id":"1704.04683","title":"RACE: Large-scale ReAding Comprehension Dataset From Examinations","created":"2017-04-15","cited_by_count":861},{"id":"1707.06209","title":"Crowdsourcing Multiple Choice Science Questions","created":"2017-07-19","cited_by_count":765},{"id":"2204.06745","title":"GPT-NeoX-20B: An Open-Source Autoregressive Language Model","created":"2022-04-14","cited_by_count":713},{"id":"2401.02385","title":"TinyLlama: An Open-Source Small Language Model","created":"2024-01-04","cited_by_count":618},{"id":"2402.04333","title":"LESS: Selecting Influential Data for Targeted Instruction Tuning","created":"2024-02-06","cited_by_count":532},{"id":"2402.00159","title":"Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Research","created":"2024-01-31","cited_by_count":483},{"id":"2305.16264","title":"Scaling Data-Constrained Language Models","created":"2023-05-25","cited_by_count":403},{"id":"2305.10429","title":"DoReMi: Optimizing Data Mixtures Speeds Up Language Model Pretraining","created":"2023-05-17","cited_by_count":399},{"id":"2312.15685","title":"What Makes Good Data for Alignment? A Comprehensive Study of Automatic Data Selection in Instruction Tuning","created":"2023-12-25","cited_by_count":364},{"id":"2211.15533","title":"The Stack: 3 TB of permissively licensed source code","created":"2022-11-20","cited_by_count":359},{"id":"2302.03169","title":"Data Selection for Language Models via Importance Resampling","created":"2023-02-06","cited_by_count":329},{"id":"2206.07137","title":"Prioritized Training on Points that are Learnable, Worth Learning, and Not Yet Learnt","created":"2022-06-14","cited_by_count":236},{"id":"2402.16827","title":"A Survey on Data Selection for Language Models","created":"2024-02-26","cited_by_count":236},{"id":"2010.15875","title":"Retrieve, Program, Repeat: Complex Knowledge Base Question Answering via Alternate Meta-learning","created":"2020-10-29","cited_by_count":216},{"id":"2403.16952","title":"Data Mixing Laws: Optimizing Data Mixtures by Predicting Language Modeling Performance","created":"2024-03-25","cited_by_count":184},{"id":"2405.14782","title":"Lessons from the Trenches on Reproducible Evaluation of Language Models","created":"2024-05-23","cited_by_count":179},{"id":"2404.07965","title":"Rho-1: Not All Tokens Are What You Need","created":"2024-04-11","cited_by_count":133},{"id":"2307.14430","title":"Skill-it! A Data-Driven Skills Framework for Understanding and Training Language Models","created":"2023-07-26","cited_by_count":121},{"id":"2403.08540","title":"Language models scale reliably with over-training and on downstream tasks","created":"2024-03-13","cited_by_count":113},{"id":"2401.12926","title":"DsDm: Model-Aware Dataset Selection with Datamodels","created":"2024-01-23","cited_by_count":105},{"id":"2406.06046","title":"MATES: Model-Aware Data Selection for Efficient Pretraining with Data Influence Models","created":"2024-06-10","cited_by_count":100},{"id":"2402.09668","title":"How to Train Data-Efficient LLMs","created":"2024-02-15","cited_by_count":93},{"id":"2310.15393","title":"DoGE: Domain Reweighting with Generalization Estimation","created":"2023-10-23","cited_by_count":90},{"id":"2212.09803","title":"Training Trajectories of Language Models Across Scales","created":"2022-12-19","cited_by_count":81},{"id":"2312.02406","title":"Efficient Online Data Mixing For Language Model Pre-Training","created":"2023-12-05","cited_by_count":77},{"id":"2404.07177","title":"Scaling Laws for Data Filtering -- Data Curation cannot be Compute Agnostic","created":"2024-04-10","cited_by_count":77},{"id":"2311.14736","title":"Data Diversity Matters for Robust Instruction Tuning","created":"2023-11-21","cited_by_count":76},{"id":"2404.14619","title":"OpenELM: An Efficient Language Model Family with Open Training and Inference Framework","created":"2024-04-22","cited_by_count":75},{"id":"2404.09937","title":"Compression Represents Intelligence Linearly","created":"2024-04-15","cited_by_count":56},{"id":"2312.10523","title":"Paloma: A Benchmark for Evaluating Language Model Fit","created":"2023-12-16","cited_by_count":54},{"id":"2403.07384","title":"SmallToLarge (S2L): Scalable Data Selection for Fine-tuning Large Language Models by Summarizing Training Trajectories of Small Models","created":"2024-03-12","cited_by_count":44},{"id":"2406.01375","title":"D-CPT Law: Domain-specific Continual Pre-Training Scaling Law for Large Language Models","created":"2024-06-03","cited_by_count":41},{"id":"2311.00913","title":"Self-Influence Guided Data Reweighting for Language Model Pre-training","created":"2023-11-02","cited_by_count":37},{"id":"2402.10430","title":"Smaller Language Models are capable of selecting Instruction-Tuning Training Data for Larger Language Models","created":"2024-02-16","cited_by_count":32},{"id":"2405.02774","title":"Get more for less: Principled Data Selection for Warming Up Fine-Tuning in LLMs","created":"2024-05-05","cited_by_count":32},{"id":"2402.07625","title":"Autonomous Data Selection with Zero-shot Generative Classifiers for Mathematical Texts","created":"2024-02-12","cited_by_count":27},{"id":"2405.14908","title":"BiMix: A Bivariate Data Mixing Law for Language Model Pretraining","created":"2024-05-23","cited_by_count":25},{"id":"2402.14526","title":"Balanced Data Sampling for Language Model Training with Clustering","created":"2024-02-22","cited_by_count":23},{"id":"2405.16684","title":"gzip Predicts Data-dependent Scaling Laws","created":"2024-05-26","cited_by_count":16},{"id":"2402.14270","title":"Take the Bull by the Horns: Hard Sample-Reweighted Continual Training Improves LLM Generalization","created":"2024-02-22","cited_by_count":12},{"id":"2405.01582","title":"Text Quality-Based Pruning for Efficient Training of Language Models","created":"2024-04-26","cited_by_count":9},{"id":"2404.01204","title":"The Fine Line: Navigating Large Language Model Pretraining with Down-streaming Capability Analysis","created":"2024-04-01","cited_by_count":4}],"next_cursor":null}