{"list":"other","total":104,"rows":[{"id":"1706.03762","title":"Attention Is All You Need","created":"2017-06-12","cited_by_count":60361},{"id":"1810.04805","title":"BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding","created":"2018-10-11","cited_by_count":40399},{"id":"2005.14165","title":"Language Models are Few-Shot Learners","created":"2020-05-28","cited_by_count":32419},{"id":"2407.21783","title":"The Llama 3 Herd of Models","created":"2024-07-31","cited_by_count":14715},{"id":"1910.10683","title":"Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer","created":"2019-10-23","cited_by_count":14599},{"id":"2110.14168","title":"Training Verifiers to Solve Math Word Problems","created":"2021-10-27","cited_by_count":10109},{"id":"2402.03300","title":"DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models","created":"2024-02-05","cited_by_count":9379},{"id":"2505.09388","title":"Qwen3 Technical Report","created":"2025-05-14","cited_by_count":8820},{"id":"2009.03300","title":"Measuring Massive Multitask Language Understanding","created":"2020-09-07","cited_by_count":7963},{"id":"2001.08361","title":"Scaling Laws for Neural Language Models","created":"2020-01-23","cited_by_count":6586},{"id":"2204.02311","title":"PaLM: Scaling Language Modeling with Pathways","created":"2022-04-05","cited_by_count":5326},{"id":"1803.05457","title":"Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge","created":"2018-03-14","cited_by_count":4886},{"id":"2104.09864","title":"RoFormer: Enhanced Transformer with Rotary Position Embedding","created":"2021-04-20","cited_by_count":4703},{"id":"2410.21276","title":"GPT-4o System Card","created":"2024-10-25","cited_by_count":4588},{"id":"1606.05250","title":"SQuAD: 100,000+ Questions for Machine Comprehension of Text","created":"2016-06-16","cited_by_count":4148},{"id":"1905.07830","title":"HellaSwag: Can a Machine Really Finish Your Sentence?","created":"2019-05-19","cited_by_count":4100},{"id":"2507.06261","title":"Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities","created":"2025-07-07","cited_by_count":3796},{"id":"2203.15556","title":"Training Compute-Optimal Large Language Models","created":"2022-03-29","cited_by_count":3019},{"id":"1705.03551","title":"TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension","created":"2017-05-09","cited_by_count":2955},{"id":"1911.11641","title":"PIQA: Reasoning about Physical Commonsense in Natural Language","created":"2019-11-26","cited_by_count":2883},{"id":"2311.12022","title":"GPQA: A Graduate-Level Google-Proof Q&A Benchmark","created":"2023-11-20","cited_by_count":2879},{"id":"1907.10902","title":"Optuna: A Next-generation Hyperparameter Optimization Framework","created":"2019-07-25","cited_by_count":2501},{"id":"2101.00027","title":"The Pile: An 800GB Dataset of Diverse Text for Language Modeling","created":"2020-12-31","cited_by_count":2411},{"id":"1905.10044","title":"BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions","created":"2019-05-24","cited_by_count":2183},{"id":"1809.02789","title":"Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering","created":"2018-09-08","cited_by_count":2164},{"id":"2206.04615","title":"Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models","created":"2022-06-09","cited_by_count":1998},{"id":"1811.00937","title":"CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge","created":"2018-11-02","cited_by_count":1822},{"id":"2002.05202","title":"GLU Variants Improve Transformer","created":"2020-02-12","cited_by_count":1804},{"id":"1910.07467","title":"Root Mean Square Layer Normalization","created":"2019-10-16","cited_by_count":1480},{"id":"2305.13245","title":"GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints","created":"2023-05-22","cited_by_count":1346},{"id":"1909.06146","title":"PubMedQA: A Dataset for Biomedical Research Question Answering","created":"2019-09-13","cited_by_count":1288},{"id":"2112.11446","title":"Scaling Language Models: Methods, Analysis & Insights from Training Gopher","created":"2021-12-08","cited_by_count":1202},{"id":"2406.17557","title":"The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale","created":"2024-06-25","cited_by_count":1160},{"id":"1606.06031","title":"The LAMBADA dataset: Word prediction requiring a broad discourse context","created":"2016-06-20","cited_by_count":992},{"id":"2103.07191","title":"Are NLP Models really able to Solve Simple Math Word Problems?","created":"2021-03-12","cited_by_count":878},{"id":"1607.01759","title":"Bag of Tricks for Efficient Text Classification","created":"2016-07-06","cited_by_count":876},{"id":"1707.06209","title":"Crowdsourcing Multiple Choice Science Questions","created":"2017-07-19","cited_by_count":765},{"id":"2306.01116","title":"The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Only","created":"2023-06-01","cited_by_count":763},{"id":"2308.03281","title":"Towards General Text Embeddings with Multi-stage Contrastive Learning","created":"2023-08-07","cited_by_count":723},{"id":"2304.06364","title":"AGIEval: A Human-Centric Benchmark for Evaluating Foundation Models","created":"2023-04-13","cited_by_count":716},{"id":"2110.08193","title":"BBQ: A Hand-Built Bias Benchmark for Question Answering","created":"2021-10-15","cited_by_count":701},{"id":"1808.07042","title":"CoQA: A Conversational Question Answering Challenge","created":"2018-08-21","cited_by_count":683},{"id":"1905.13319","title":"MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalisms","created":"2019-05-30","cited_by_count":662},{"id":"1705.04146","title":"Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problems","created":"2017-05-11","cited_by_count":650},{"id":"2401.02954","title":"DeepSeek LLM: Scaling Open-Source Language Models with Longtermism","created":"2024-01-05","cited_by_count":642},{"id":"2309.07597","title":"C-Pack: Packed Resources For General Chinese Embeddings","created":"2023-09-14","cited_by_count":578},{"id":"1907.09190","title":"ELI5: Long Form Question Answering","created":"2019-07-22","cited_by_count":575},{"id":"2201.11990","title":"Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model","created":"2022-01-28","cited_by_count":559},{"id":"1911.00359","title":"CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Data","created":"2019-11-01","cited_by_count":548},{"id":"2305.16264","title":"Scaling Data-Constrained Language Models","created":"2023-05-25","cited_by_count":403},{"id":"2305.10429","title":"DoReMi: Optimizing Data Mixtures Speeds Up Language Model Pretraining","created":"2023-05-17","cited_by_count":399},{"id":"2406.11794","title":"DataComp-LM: In search of the next generation of training sets for language models","created":"2024-06-17","cited_by_count":398},{"id":"1904.01557","title":"Analysing Mathematical Reasoning Abilities of Neural Models","created":"2019-04-02","cited_by_count":335},{"id":"2406.11931","title":"DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligence","created":"2024-06-17","cited_by_count":331},{"id":"2302.03169","title":"Data Selection for Language Models via Importance Resampling","created":"2023-02-06","cited_by_count":329},{"id":"2303.09540","title":"SemDeDup: Data-efficient learning at web-scale through semantic deduplication","created":"2023-03-16","cited_by_count":303},{"id":"2203.03466","title":"Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer","created":"2022-03-07","cited_by_count":288},{"id":"1812.06162","title":"An Empirical Model of Large-Batch Training","created":"2018-12-14","cited_by_count":281},{"id":"2309.17425","title":"Data Filtering Networks","created":"2023-09-29","cited_by_count":250},{"id":"2402.16827","title":"A Survey on Data Selection for Language Models","created":"2024-02-26","cited_by_count":236},{"id":"2407.19669","title":"mGTE: Generalized Long-Context Text Representation and Reranking Models for Multilingual Text Retrieval","created":"2024-07-29","cited_by_count":236},{"id":"2010.15875","title":"Retrieve, Program, Repeat: Complex Knowledge Base Question Answering via Alternate Meta-learning","created":"2020-10-29","cited_by_count":216},{"id":"2309.14322","title":"Small-scale proxies for large-scale Transformer training instabilities","created":"2023-09-25","cited_by_count":183},{"id":"2407.01492","title":"RegMix: Data Mixture as Regression for Language Model Pre-training","created":"2024-07-01","cited_by_count":177},{"id":"2401.16380","title":"Rephrasing the Web: A Recipe for Compute and Data-Efficient Language Modeling","created":"2024-01-29","cited_by_count":161},{"id":"2412.02595","title":"Nemotron-CC: Transforming Common Crawl into a Refined Long-Horizon Pretraining Dataset","created":"2024-12-03","cited_by_count":160},{"id":"2402.09739","title":"QuRating: Selecting High-Quality Data for Training Language Models","created":"2024-02-15","cited_by_count":157},{"id":"2404.07965","title":"Rho-1: Not All Tokens Are What You Need","created":"2024-04-11","cited_by_count":133},{"id":"2307.14430","title":"Skill-it! A Data-Driven Skills Framework for Understanding and Training Language Models","created":"2023-07-26","cited_by_count":121},{"id":"2403.08540","title":"Language models scale reliably with over-training and on downstream tasks","created":"2024-03-13","cited_by_count":113},{"id":"2406.06046","title":"MATES: Model-Aware Data Selection for Efficient Pretraining with Data Influence Models","created":"2024-06-10","cited_by_count":100},{"id":"2402.09668","title":"How to Train Data-Efficient LLMs","created":"2024-02-15","cited_by_count":93},{"id":"2310.15393","title":"DoGE: Domain Reweighting with Generalization Estimation","created":"2023-10-23","cited_by_count":90},{"id":"2405.20541","title":"Perplexed by Perplexity: Perplexity-Based Data Pruning With Small Reference Models","created":"2024-05-30","cited_by_count":84},{"id":"2312.02406","title":"Efficient Online Data Mixing For Language Model Pre-Training","created":"2023-12-05","cited_by_count":77},{"id":"2404.07177","title":"Scaling Laws for Data Filtering -- Data Curation cannot be Compute Agnostic","created":"2024-04-10","cited_by_count":77},{"id":"2502.10341","title":"Organize the Web: Constructing Domains Enhances Pre-Training Data Curation","created":"2025-02-14","cited_by_count":76},{"id":"2410.21676","title":"How Does Critical Batch Size Scale in Pre-training?","created":"2024-10-29","cited_by_count":73},{"id":"2404.10102","title":"Chinchilla Scaling: A replication attempt","created":"2024-04-15","cited_by_count":72},{"id":"2502.08606","title":"Distillation Scaling Laws","created":"2025-02-12","cited_by_count":67},{"id":"2504.13161","title":"Nemotron-CLIMB: CLustering-based Iterative Data Mixture Bootstrapping for Language Model Pre-training","created":"2025-04-17","cited_by_count":66},{"id":"2412.04506","title":"Arctic-Embed 2.0: Multilingual Retrieval Without Compromise","created":"2024-12-03","cited_by_count":58},{"id":"2406.03476","title":"Does your data spark joy? Performance gains from domain upsampling at the end of training","created":"2024-06-05","cited_by_count":52},{"id":"2507.09404","title":"Scaling Laws for Optimal Data Mixtures","created":"2025-07-12","cited_by_count":52},{"id":"2410.11820","title":"Adaptive Data Optimization: Dynamic Sample Selection with Scaling Laws","created":"2024-10-15","cited_by_count":47},{"id":"2409.05816","title":"Improving Pretraining Data Using Perplexity Correlations","created":"2024-09-09","cited_by_count":46},{"id":"2406.04391","title":"Why Has Predicting Downstream Capabilities of Frontier AI Models with Scale Remained Elusive?","created":"2024-06-06","cited_by_count":40},{"id":"2504.11393","title":"DataDecide: How to Predict Best Pretraining Data with Small Experiments","created":"2025-04-15","cited_by_count":40},{"id":"2410.07064","title":"Data Selection via Optimal Control for Language Models","created":"2024-10-09","cited_by_count":38},{"id":"2412.04403","title":"Establishing Task Scaling Laws via Compute-Efficient Model Ladders","created":"2024-12-05","cited_by_count":35},{"id":"2411.05735","title":"Aioli: A Unified Optimization Framework for Language Model Data Mixing","created":"2024-11-08","cited_by_count":34},{"id":"2309.08632","title":"Pretraining on the Test Set Is All You Need","created":"2023-09-13","cited_by_count":33},{"id":"2503.00808","title":"Predictive Data Selection: The Data That Predicts Is the Data That Teaches","created":"2025-03-02","cited_by_count":21},{"id":"2406.10670","title":"CoLoR-Filter: Conditional Loss Reduction Filtering for Targeted Language Model Pre-training","created":"2024-06-15","cited_by_count":20},{"id":"2410.03735","title":"Task-Adaptive Pretrained Language Models via Clustered-Importance Sampling","created":"2024-09-30","cited_by_count":18},{"id":"2405.16684","title":"gzip Predicts Data-dependent Scaling Laws","created":"2024-05-26","cited_by_count":16},{"id":"2502.12120","title":"LLMs on the Line: Data Determines Loss-to-Loss Scaling Laws","created":"2025-02-17","cited_by_count":16},{"id":"2411.12925","title":"Loss-to-Loss Prediction: Scaling Laws for All Datasets","created":"2024-11-19","cited_by_count":15},{"id":"2411.06068","title":"Zyda-2: a 5 Trillion Token High-Quality Dataset","created":"2024-11-09","cited_by_count":13},{"id":"2505.17895","title":"DataRater: Meta-Learned Dataset Curation","created":"2025-05-23","cited_by_count":13}],"next_cursor":"100"}