{"list":"other","total":57,"rows":[{"id":"1910.10683","title":"Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer","created":"2019-10-23","cited_by_count":14599},{"id":"2505.09388","title":"Qwen3 Technical Report","created":"2025-05-14","cited_by_count":8820},{"id":"2107.03374","title":"Evaluating Large Language Models Trained on Code","created":"2021-07-07","cited_by_count":8320},{"id":"2009.03300","title":"Measuring Massive Multitask Language Understanding","created":"2020-09-07","cited_by_count":7963},{"id":"1803.05457","title":"Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge","created":"2018-03-14","cited_by_count":4886},{"id":"2312.00752","title":"Mamba: Linear-Time Sequence Modeling with Selective State Spaces","created":"2023-12-01","cited_by_count":4718},{"id":"1905.07830","title":"HellaSwag: Can a Machine Really Finish Your Sentence?","created":"2019-05-19","cited_by_count":4100},{"id":"2108.07732","title":"Program Synthesis with Large Language Models","created":"2021-08-16","cited_by_count":3637},{"id":"1609.07843","title":"Pointer Sentinel Mixture Models","created":"2016-09-26","cited_by_count":3096},{"id":"2101.00027","title":"The Pile: An 800GB Dataset of Diverse Text for Language Modeling","created":"2020-12-31","cited_by_count":2411},{"id":"2309.17453","title":"Efficient Streaming Language Models with Attention Sinks","created":"2023-09-29","cited_by_count":2186},{"id":"1901.02860","title":"Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context","created":"2019-01-09","cited_by_count":1850},{"id":"2304.01373","title":"Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling","created":"2023-04-03","cited_by_count":1786},{"id":"2206.14858","title":"Solving Quantitative Reasoning Problems with Language Models","created":"2022-06-29","cited_by_count":1392},{"id":"2305.13245","title":"GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints","created":"2023-05-22","cited_by_count":1346},{"id":"2404.06654","title":"RULER: What's the Real Context Size of Your Long-Context Language Models?","created":"2024-04-09","cited_by_count":1346},{"id":"2108.12409","title":"Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation","created":"2021-08-27","cited_by_count":1103},{"id":"2306.15595","title":"Extending Context Window of Large Language Models via Positional Interpolation","created":"2023-06-27","cited_by_count":740},{"id":"2402.00838","title":"OLMo: Accelerating the Science of Language Models","created":"2024-02-01","cited_by_count":613},{"id":"2309.00071","title":"YaRN: Efficient Context Window Extension of Large Language Models","created":"2023-08-31","cited_by_count":601},{"id":"2412.06464","title":"Gated Delta Networks: Improving Mamba2 with Delta Rule","created":"2024-12-09","cited_by_count":532},{"id":"2402.00159","title":"Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Research","created":"2024-01-31","cited_by_count":483},{"id":"2104.02112","title":"Efficient Attentions for Long Document Summarization","created":"2021-04-05","cited_by_count":345},{"id":"2305.19466","title":"The Impact of Positional Encoding on Length Generalization in Transformers","created":"2023-05-31","cited_by_count":327},{"id":"2402.13753","title":"LongRoPE: Extending LLM Context Window Beyond 2 Million Tokens","created":"2024-02-21","cited_by_count":313},{"id":"2309.16039","title":"Effective Long-Context Scaling of Foundation Models","created":"2023-09-27","cited_by_count":303},{"id":"2505.06708","title":"Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Free","created":"2025-05-10","cited_by_count":283},{"id":"1812.06162","title":"An Empirical Model of Large-Batch Training","created":"2018-12-14","cited_by_count":281},{"id":"2404.15574","title":"Retrieval Head Mechanistically Explains Long-Context Factuality","created":"2024-04-24","cited_by_count":232},{"id":"2410.10781","title":"When Attention Sink Emerges in Language Models: An Empirical View","created":"2024-10-14","cited_by_count":208},{"id":"2402.10171","title":"Data Engineering for Scaling Language Models to 128K Context","created":"2024-02-15","cited_by_count":186},{"id":"2306.12929","title":"Quantizable Transformers: Removing Outliers by Helping Attention Heads Do Nothing","created":"2023-06-22","cited_by_count":151},{"id":"2410.02660","title":"How to Train Long-Context Language Models (Effectively)","created":"2024-10-03","cited_by_count":146},{"id":"2212.10554","title":"A Length-Extrapolatable Transformer","created":"2022-12-20","cited_by_count":127},{"id":"2504.02732","title":"Why do LLMs attend to the first token?","created":"2025-04-03","cited_by_count":127},{"id":"2207.00220","title":"Pile of Law: Learning Responsible Data Filtering from the Law and a 256GB Open-Source Legal Dataset","created":"2022-07-01","cited_by_count":113},{"id":"2410.21676","title":"How Does Critical Batch Size Scale in Pre-training?","created":"2024-10-29","cited_by_count":73},{"id":"2311.12351","title":"Advancing Transformer Architecture in Long-Context Large Language Models: A Comprehensive Survey","created":"2023-11-21","cited_by_count":72},{"id":"2410.23771","title":"What is Wrong with Perplexity for Long-context Language Modeling?","created":"2024-10-31","cited_by_count":69},{"id":"2312.10523","title":"Paloma: A Benchmark for Evaluating Language Model Fit","created":"2023-12-16","cited_by_count":54},{"id":"2404.12224","title":"Length Generalization of Causal Transformers without Position Encoding","created":"2024-04-18","cited_by_count":47},{"id":"2501.18795","title":"Rope to Nope and Back Again: A New Hybrid Attention Strategy","created":"2025-01-30","cited_by_count":47},{"id":"2404.08801","title":"Megalodon: Efficient LLM Pretraining and Inference with Unlimited Context Length","created":"2024-04-12","cited_by_count":46},{"id":"2406.04391","title":"Why Has Predicting Downstream Capabilities of Frontier AI Models with Scale Remained Elusive?","created":"2024-06-06","cited_by_count":40},{"id":"2504.11393","title":"DataDecide: How to Predict Best Pretraining Data with Small Experiments","created":"2025-04-15","cited_by_count":40},{"id":"2502.02732","title":"Peri-LN: Revisiting Normalization Layer in the Transformer Architecture","created":"2025-02-04","cited_by_count":34},{"id":"2210.07370","title":"M2D2: A Massively Multi-domain Language Modeling Dataset","created":"2022-10-13","cited_by_count":26},{"id":"2409.12181","title":"A Controlled Study on Long Context Extension and Generalization in LLMs","created":"2024-09-18","cited_by_count":26},{"id":"2503.05804","title":"Holistically Evaluating the Environmental Impact of Creating Language Models","created":"2025-03-03","cited_by_count":26},{"id":"2508.13144","title":"Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation","created":"2025-08-18","cited_by_count":26},{"id":"2409.00509","title":"LongRecipe: Recipe for Efficient Long Context Generalization in Large Language Models","created":"2024-08-31","cited_by_count":23},{"id":"2505.23971","title":"Critical Batch Size Revisited: A Simple Empirical Approach to Large-Batch Language Model Training","created":"2025-05-29","cited_by_count":23},{"id":"2601.22966","title":"A Unified View of Attention and Residual Sinks: Outlier-Driven Rescaling is Essential for Transformer Training","created":"2026-01-30","cited_by_count":23},{"id":"2406.00605","title":"LongSkywork: A Training Recipe for Efficiently Extending Context Length in Large Language Models","created":"2024-06-02","cited_by_count":19},{"id":"2503.03862","title":"Not-Just-Scaling Laws: Towards a Better Understanding of the Downstream Impact of Language Model Design Decisions","created":"2025-03-05","cited_by_count":12},{"id":"2510.24397","title":"APTBench: Benchmarking Agentic Potential of Base LLMs During Pre-Training","created":"2025-10-28","cited_by_count":11},{"id":"2311.16620","title":"On the Long Range Abilities of Transformers","created":"2023-11-28","cited_by_count":10}],"next_cursor":null}