{"total":119,"rows":[{"id":"2306.01116","title":"The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Only","created":"2023-06-01","cited_by_count":763,"area":null},{"id":"2311.16867","title":"The Falcon Series of Open Language Models","created":"2023-11-28","cited_by_count":535,"area":null},{"id":"2305.16264","title":"Scaling Data-Constrained Language Models","created":"2023-05-25","cited_by_count":403,"area":null},{"id":"2307.10169","title":"Challenges and Applications of Large Language Models","created":"2023-07-19","cited_by_count":254,"area":null},{"id":"2505.24760","title":"REASONING GYM: Reasoning Environments for Reinforcement Learning with Verifiable Rewards","created":"2025-05-30","cited_by_count":130,"area":null},{"id":"2311.09807","title":"The Curious Decline of Linguistic Diversity: Training Language Models on Synthetic Text","created":"2023-11-16","cited_by_count":100,"area":null},{"id":"2306.15546","title":"When Foundation Model Meets Federated Learning: Motivations, Challenges, and Future Directions","created":"2023-06-27","cited_by_count":84,"area":null},{"id":"2506.10943","title":"Self-Adapting Language Models","created":"2025-06-12","cited_by_count":84,"area":null},{"id":"2311.16822","title":"Large Language Models Suffer From Their Own Output: An Analysis of the Self-Consuming Training Loop","created":"2023-11-28","cited_by_count":79,"area":null},{"id":"2309.16575","title":"A Benchmark for Learning to Translate a New Language from One Grammar Book","created":"2023-09-28","cited_by_count":75,"area":null},{"id":"2409.07431","title":"Synthetic continued pretraining","created":"2024-09-11","cited_by_count":60,"area":null},{"id":"2312.06353","title":"Federated Full-Parameter Tuning of Billion-Sized Language Models with Communication Cost under 18 Kilobytes","created":"2023-12-11","cited_by_count":55,"area":null},{"id":"2402.02622","title":"DenseFormer: Enhancing Information Flow in Transformers via Depth Weighted Averaging","created":"2024-02-04","cited_by_count":53,"area":null},{"id":"2502.19249","title":"Between Circuits and Chomsky: Pre-pretraining on Formal Languages Imparts Linguistic Biases","created":"2025-02-26","cited_by_count":37,"area":null},{"id":"2502.14425","title":"A Survey on Data Contamination for Large Language Models","created":"2025-02-20","cited_by_count":35,"area":null},{"id":"2311.09227","title":"Open-Sourcing Highly Capable Foundation Models: An evaluation of risks, benefits, and alternative methods for pursuing open-source objectives","created":"2023-09-29","cited_by_count":31,"area":null},{"id":"2509.14786","title":"Pre-training under infinite compute","created":"2025-09-18","cited_by_count":29,"area":null},{"id":"2404.09173","title":"TransformerFAM: Feedback attention is working memory","created":"2024-04-14","cited_by_count":22,"area":null},{"id":"2602.05400","title":"OPUS: Towards Efficient and Principled Data Selection in Large Language Model Pre-training in Every Iteration","created":"2026-02-05","cited_by_count":21,"area":null},{"id":"2405.15815","title":"A social path to human-like artificial intelligence","created":"2024-05-22","cited_by_count":20,"area":null},{"id":"2505.23725","title":"MuLoCo: Muon is a practical inner optimizer for DiLoCo","created":"2025-05-29","cited_by_count":18,"area":null},{"id":"2405.05990","title":"Special Characters Attack: Toward Scalable Training Data Extraction From Large Language Models","created":"2024-05-09","cited_by_count":17,"area":null},{"id":"2504.12427","title":"Position: The Most Expensive Part of an LLM should be its Training Data","created":"2025-04-16","cited_by_count":16,"area":null},{"id":"2311.01550","title":"Market Concentration Implications of Foundation Models","created":"2023-11-02","cited_by_count":15,"area":null},{"id":"2603.10055","title":"Training Language Models via Neural Cellular Automata","created":"2026-03-09","cited_by_count":15,"area":null},{"id":"2309.07207","title":"EarthPT: a time series foundation model for Earth Observation","created":"2023-09-13","cited_by_count":14,"area":null},{"id":"2405.10799","title":"Training Compute Thresholds: Features and Functions in AI Regulation","created":"2024-05-17","cited_by_count":14,"area":null},{"id":"2508.09494","title":"Learning Facts at Scale with Active Reading","created":"2025-08-13","cited_by_count":13,"area":null},{"id":"2601.21571","title":"Shaping capabilities with token-level data filtering","created":"2026-01-29","cited_by_count":13,"area":null},{"id":"2402.14270","title":"Take the Bull by the Horns: Hard Sample-Reweighted Continual Training Improves LLM Generalization","created":"2024-02-22","cited_by_count":12,"area":null},{"id":"2506.22696","title":"Residual Matrix Transformers: Scaling the Size of the Residual Stream","created":"2025-06-28","cited_by_count":12,"area":null},{"id":"2602.11698","title":"SpiralFormer: Looped Transformers Can Learn Hierarchical Dependencies via Multi-Resolution Recursion","created":"2026-02-12","cited_by_count":12,"area":null},{"id":"2506.02961","title":"FlowerTune: A Cross-Domain Benchmark for Federated Fine-Tuning of Large Language Models","created":"2025-06-03","cited_by_count":11,"area":null},{"id":"2407.08642","title":"Towards Building Specialized Generalist AI with System 1 and System 2 Fusion","created":"2024-07-11","cited_by_count":8,"area":null},{"id":"2410.20796","title":"Rephrasing natural text data with different languages and quality levels for Large Language Model pre-training","created":"2024-10-28","cited_by_count":8,"area":null},{"id":"2507.14240","title":"HuggingGraph: Understanding the Supply Chain of LLM Ecosystem","created":"2025-07-17","cited_by_count":8,"area":null},{"id":"2510.07739","title":"MeSH: Memory-as-State-Highways for Recursive Transformers","created":"2025-10-09","cited_by_count":8,"area":null},{"id":"2605.19407","title":"A Bitter Lesson for Data Filtering","created":"2026-05-19","cited_by_count":8,"area":null},{"id":"2310.14579","title":"FedSplitX: Federated Split Learning for Computationally-Constrained Heterogeneous Clients","created":"2023-10-23","cited_by_count":7,"area":null},{"id":"2603.18534","title":"Data-efficient pre-training by scaling synthetic megadocs","created":"2026-03-19","cited_by_count":7,"area":null},{"id":"2501.07314","title":"FinerWeb-10BT: Refining Web Data with LLM-Based Line-Level Filtering","created":"2025-01-13","cited_by_count":6,"area":null},{"id":"2508.14927","title":"AI Testing Should Account for Sophisticated Strategic Behaviour","created":"2025-08-19","cited_by_count":6,"area":null},{"id":"2510.10681","title":"RePro: Training Language Models to Faithfully Recycle the Web for Pretraining","created":"2025-10-12","cited_by_count":6,"area":null},{"id":"2307.06933","title":"FDAPT: Federated Domain-adaptive Pre-training for Language Models","created":"2023-07-12","cited_by_count":5,"area":null},{"id":"2402.14652","title":"Cleaner Pretraining Corpus Curation with Neural Web Scraping","created":"2024-02-22","cited_by_count":5,"area":null},{"id":"2412.13860","title":"Domain-adaptative Continual Learning for Low-resource Tasks: Evaluation on Nepali","created":"2024-12-18","cited_by_count":5,"area":null},{"id":"2503.01630","title":"Machine Learners Should Acknowledge the Legal Implications of Large Language Models as Personal Data","created":"2025-03-03","cited_by_count":5,"area":null},{"id":"2505.00985","title":"Position: Enough of Scaling LLMs! Lets Focus on Downscaling","created":"2025-05-02","cited_by_count":5,"area":null},{"id":"2506.20057","title":"Universal pre-training by iterated random computation","created":"2025-06-24","cited_by_count":5,"area":null},{"id":"2508.03097","title":"VFLAIR-LLM: A Comprehensive Framework and Benchmark for Split Learning of LLMs","created":"2025-08-05","cited_by_count":5,"area":null},{"id":"2311.05020","title":"First Tragedy, then Parse: History Repeats Itself in the New Era of Large Language Models","created":"2023-11-08","cited_by_count":4,"area":null},{"id":"2403.04529","title":"Enhancing Data Quality in Federated Fine-Tuning of Foundation Models","created":"2024-03-07","cited_by_count":4,"area":null},{"id":"2405.09597","title":"When AI Eats Itself: On the Caveats of AI Autophagy","created":"2024-05-15","cited_by_count":4,"area":null},{"id":"2504.17421","title":"Towards Harnessing the Collaborative Power of Large and Small Models for Domain Tasks","created":"2025-04-24","cited_by_count":4,"area":null},{"id":"2508.08011","title":"Optimal Transport Depth Up-Scaling","created":"2025-08-11","cited_by_count":4,"area":null},{"id":"2605.05076","title":"High-Dimensional Statistics: Reflections on Progress and Open Problems","created":"2026-05-06","cited_by_count":4,"area":null},{"id":"2311.00382","title":"Will Code Remain a Relevant User Interface for End-User Programming with Generative AI Models?","created":"2023-11-01","cited_by_count":3,"area":null},{"id":"2403.15955","title":"Finding needles in a haystack: A Black-Box Approach to Invisible Watermark Detection","created":"2024-03-23","cited_by_count":3,"area":null},{"id":"2412.03824","title":"Towards Data Governance of Frontier AI Models","created":"2024-12-05","cited_by_count":3,"area":null},{"id":"2503.08085","title":"PRISM: Privacy-Preserving Improved Stochastic Masking for Federated Generative Models","created":"2025-03-11","cited_by_count":3,"area":null},{"id":"2507.17578","title":"Synthetic Voice Data for Automatic Speech Recognition in African Languages","created":"2025-07-23","cited_by_count":3,"area":null},{"id":"2511.06305","title":"Setting $\\varepsilon$ is not the Issue in Differential Privacy","created":"2025-11-09","cited_by_count":3,"area":null},{"id":"2512.08946","title":"Large Language Models as Search Engines: Societal Challenges","created":"2025-11-24","cited_by_count":3,"area":null},{"id":"2603.14420","title":"Data Darwinism Part II: DataEvolve -- AI can Autonomously Evolve Pretraining Data Curation","created":"2026-03-15","cited_by_count":3,"area":null},{"id":"2606.06888","title":"Data-Constrained Language Model Pretraining: Improved Regularization and Scaling Laws","created":"2026-06-05","cited_by_count":3,"area":null},{"id":"2607.01218","title":"The State-Prediction Separation Hypothesis","created":"2026-07-01","cited_by_count":3,"area":null},{"id":"2411.16917","title":"Are Transformers Truly Foundational for Robotics?","created":"2024-11-25","cited_by_count":2,"area":null},{"id":"2501.17980","title":"Limits to AI Growth: The Ecological and Social Consequences of Scaling","created":"2025-01-29","cited_by_count":2,"area":null},{"id":"2504.05563","title":"Do Data Valuations Make Good Data Prices?","created":"2025-04-07","cited_by_count":2,"area":null},{"id":"2504.06843","title":"Integrating Cognitive Processing Signals into Language Models: A Review of Advances, Applications and Future Directions","created":"2025-04-09","cited_by_count":2,"area":null},{"id":"2508.01317","title":"LinkQA: Synthesizing Diverse QA from Multiple Seeds Strongly Linked by Knowledge Points","created":"2025-08-02","cited_by_count":2,"area":null},{"id":"2509.11937","title":"MMORE: Massive Multimodal Open RAG & Extraction","created":"2025-09-15","cited_by_count":2,"area":null},{"id":"2605.17849","title":"Generating Pretraining Tokens from Organic Data for Data-Bound Scaling","created":"2026-05-18","cited_by_count":2,"area":null},{"id":"2606.01155","title":"When Data Is Scarce: Scaling Sparse Language Models with Repeated Training","created":"2026-05-31","cited_by_count":2,"area":null},{"id":"2607.25271","title":"Bridging Compute- and Data-Optimal Pretraining","created":"2026-07-28","cited_by_count":2,"area":null},{"id":"2609.19107","title":"How Model Growth, Recursion, and Boundary Operators Influence Scaling Exponents","created":"2026-09-16","cited_by_count":2,"area":null},{"id":"2310.10955","title":"A State-Vector Framework for Dataset Effects","created":"2023-10-17","cited_by_count":1,"area":null},{"id":"2310.17972","title":"EcoLearn: Optimizing the Carbon Footprint of Federated Learning","created":"2023-10-27","cited_by_count":1,"area":null},{"id":"2402.18329","title":"Robust Synthetic Data-Driven Detection of Living-Off-the-Land Reverse Shells","created":"2024-02-28","cited_by_count":1,"area":null},{"id":"2404.19484","title":"More Compute Is What You Need","created":"2024-04-30","cited_by_count":1,"area":null},{"id":"2405.13030","title":"Crowdsourcing with Enhanced Data Quality Assurance: An Efficient Approach to Mitigate Resource Scarcity Challenges in Training Large Language Models for Healthcare","created":"2024-05-16","cited_by_count":1,"area":null},{"id":"2410.11444","title":"A Theoretical Survey on Foundation Models","created":"2024-10-15","cited_by_count":1,"area":null},{"id":"2410.20522","title":"Props for Machine-Learning Security","created":"2024-10-27","cited_by_count":1,"area":null},{"id":"2502.15719","title":"Governing AI Beyond the Pretraining Frontier","created":"2025-01-27","cited_by_count":1,"area":null},{"id":"2503.17414","title":"Opportunities and Challenges of Frontier Data Governance With Synthetic Data","created":"2025-03-21","cited_by_count":1,"area":null},{"id":"2504.17321","title":"Dargana: fine-tuning EarthPT for dynamic tree canopy mapping from space","created":"2025-04-24","cited_by_count":1,"area":null},{"id":"2507.04432","title":"Reconstructing Biological Pathways by Applying Selective Incremental Learning to (Very) Small Language Models","created":"2025-07-06","cited_by_count":1,"area":null},{"id":"2507.23440","title":"Self-Foveate: Enhancing Diversity and Difficulty of Synthesized Instructions from Unsupervised Text via Multi-Level Foveation","created":"2025-07-31","cited_by_count":1,"area":null},{"id":"2509.11176","title":"Differentially-private text generation degrades output language quality","created":"2025-09-14","cited_by_count":1,"area":null},{"id":"2511.01253","title":"Quantum Deep Learning Still Needs a Quantum Leap","created":"2025-11-03","cited_by_count":1,"area":null},{"id":"2601.04474","title":"Computational Compliance for AI Regulation: Blueprint for a New Research Domain","created":"2026-01-08","cited_by_count":1,"area":null},{"id":"2603.28386","title":"COvolve: Adversarial Co-Evolution of Large-Language-Model-Generated Policies and Environments via Two-Player Zero-Sum Game","created":"2026-03-30","cited_by_count":1,"area":null},{"id":"2606.13892","title":"Crypto x AI, AI x Crypto: A Survey","created":"2026-06-11","cited_by_count":1,"area":null},{"id":"2608.13482","title":"Synthetic Persona Pretraining: Alignment from Token Zero","created":"2026-08-13","cited_by_count":1,"area":null},{"id":"2609.17699","title":"NeMo Data Designer: An Extensible Framework for Multimodal Synthetic Data Generation","created":"2026-09-15","cited_by_count":1,"area":null},{"id":"2609.36054","title":"What if automating AI R&D triggers an intelligence explosion?","created":"2026-09-28","cited_by_count":1,"area":null},{"id":"2403.03920","title":"Enhancing Instructional Quality: Leveraging Computer-Assisted Textual Analysis to Generate In-Depth Insights from Educational Artifacts","created":"2024-03-06","cited_by_count":0,"area":null},{"id":"2405.16728","title":"Towards Multi-Task Multi-Modal Models: A Video Generative Perspective","created":"2024-05-26","cited_by_count":0,"area":null},{"id":"2410.18194","title":"ZIP-FIT: Embedding-Free Data Selection via Compression-Based Alignment","created":"2024-10-23","cited_by_count":0,"area":null},{"id":"2502.01806","title":"Toward Neurosymbolic Program Comprehension","created":"2025-02-03","cited_by_count":0,"area":null}],"next_cursor":"100"}