mlsanityData to 14 Oct 2026

Rho-1: Not All Tokens Are What You Need

arXiv 2404.07965 ↗First posted 11 Apr 2024cs.CL cs.AIcited by 133

Not on the map. Why?

Cited by 37 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Previous language model pre-training methods have uniformly applied a next-token prediction loss to all training tokens. Challenging this norm, we posit that "9l training". Our initial analysis examines token-level training dynamics of language model, revealing distinct loss patterns for different tokens. Leveraging these insights, we introduce a new language model called Rho-1. Unlike traditional LMs that learn to predict every next token in a corpus, Rho-1 employs Selective Language Modeling (SLM), which selectively trains on useful tokens that aligned with the desired distribution. This approach involves scoring pretraining tokens using a reference model, and then training the language model with a focused loss on tokens with higher scores. When continual pretraining on 15B OpenWebMath corpus, Rho-1 yields an absolute improvement in few-shot accuracy of up to 30% in 9 math tasks. After fine-tuning, Rho-1-1B and 7B achieved state-of-the-art results of 40.6% and 51.8% on MATH dataset, respectively - matching DeepSeekMath with only 3% of the pretraining tokens. Furthermore, when continual pretraining on 80B general tokens, Rho-1 achieves 6.8% average enhancement across 15 diverse tasks, increasing both efficiency and performance of the language model pre-training.

Key facts. Cited by 133 papers in the map (data through 14 Oct 2026). Builds on 101 of its 118 references that are in the map. First posted on arXiv 11 Apr 2024.

Builds on

101 of 118 references.

References · 101

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2022Training language models to follow instructions with human feedbackcited by 15,971
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  6. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  7. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  8. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  9. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  10. 2020Scaling Laws for Neural Language Modelscited by 6,586
  11. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  12. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
38 more
  1. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  2. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
  3. 2023Let's Verify Step by Stepcited by 3,789
  4. 2021Program Synthesis with Large Language Modelscited by 3,637
  5. 2023Qwen Technical Reportcited by 3,344
  6. 2023Mistral 7Bcited by 3,164
  7. 2022Training Compute-Optimal Large Language Modelscited by 3,019
  8. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,883
  9. 2022Emergent Abilities of Large Language Modelscited by 2,512
  10. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,183
  11. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,164
  12. 2023Code Llama: Open Foundation Models for Codecited by 2,139
  13. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,807
  14. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,392
  15. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,181
  16. 2015Prioritized Experience Replaycited by 1,099
  17. 2024Gemma: Open Models Based on Gemini Research and Technologycited by 916
  18. 2019Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methodscited by 913
  19. 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 878
  20. 2022Quantifying Memorization Across Neural Language Modelscited by 763
  21. 2023MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Modelscited by 735
  22. 2023AGIEval: A Human-Centric Benchmark for Evaluating Foundation Modelscited by 716
  23. 2021Deduplicating Training Data Makes Language Models Bettercited by 714
  24. 2019Deep Double Descent: Where Bigger Models and More Data Hurtcited by 672
  25. 2019MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalismscited by 662
  26. 2022PAL: Program-aided Language Modelscited by 652
  27. 2024DeepSeek LLM: Scaling Open-Source Language Models with Longtermismcited by 642
  28. 2022Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasetscited by 629
  29. 2022Contrastive Decoding: Open-ended Text Generation as Optimizationcited by 609
  30. 2023Textbooks Are All You Needcited by 569
  31. 2019CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Datacited by 548
  32. 2024LESS: Selecting Influential Data for Targeted Instruction Tuningcited by 532
  33. 2021Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpuscited by 492
  34. 2020TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languagescited by 488
  35. 2023MAmmoTH: Building Math Generalist Models through Hybrid Instruction Tuningcited by 487
  36. 2023Scaling Data-Constrained Language Modelscited by 403
  37. 2023DoReMi: Optimizing Data Mixtures Speeds Up Language Model Pretrainingcited by 399
  38. 2020Generative Language Modeling for Automated Theorem Provingcited by 388

50 most cited of 101 All 101 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 42; 2025: 62; 2026: 29.

133 papers.

Sort:

By area:

  1. 2025OpenThoughts: Data Recipes for Reasoning Modelscited by 304
  2. 2024Step-DPO: Step-wise Preference Optimization for Long-chain Reasoning of LLMscited by 273
  3. 2024Continual Learning of Large Language Models: A Comprehensive Surveycited by 234
  4. 2025MedXpertQA: Benchmarking Expert-Level Medical Reasoning and Understandingcited by 224
  5. 2024VinePPO: Refining Credit Assignment in RL Training of LLMscited by 183
  6. 2024RegMix: Data Mixture as Regression for Language Model Pre-trainingcited by 177
  7. 2024Beyond Autoregression: Discrete Diffusion for Complex Reasoning and Planningcited by 135
  8. 2024MATES: Model-Aware Data Selection for Efficient Pretraining with Data Influence Modelscited by 100
  9. 2024A Comprehensive Survey of Scientific Large Language Models and Their Applications in Scientific Discoverycited by 96
  10. 2024Critical Tokens Matter: Token-Level Contrastive Estimation Enhances LLM's Reasoning Capabilitycited by 83
  11. 2025Segment Policy Optimization: Effective Segment-Level Credit Assignment in RL for Large Language Modelscited by 75
  12. 2024TableGPT2: A Large Multimodal Model with Tabular Data Integrationcited by 72
38 more
  1. 2024Asynchronous RLHF: Faster and More Efficient Off-Policy RL for Language Modelscited by 70
  2. 2024What is Wrong with Perplexity for Long-context Language Modeling?cited by 69
  3. 2024Towards Lifelong Learning of Large Language Models: A Surveycited by 64
  4. 2024A Survey of Mathematical Reasoning in the Era of Multimodal Large Language Model: Benchmark, Method & Challengescited by 55
  5. 2025A Survey on Large Language Models for Mathematical Reasoningcited by 52
  6. 2025Demystifying Reasoning Dynamics with Mutual Information: Thinking Tokens are Information Peaks in LLM Reasoningcited by 51
  7. 2024Be like a Goldfish, Don't Memorize! Mitigating Memorization in Generative LLMscited by 50
  8. 2024Is Your Model Really A Good Math Reasoner? Evaluating Mathematical Reasoning with Checklistcited by 48
  9. 2024JiuZhang3.0: Efficiently Improving Mathematical Reasoning by Training Small Data Synthesis Modelscited by 45
  10. 2024Data Selection via Optimal Control for Language Modelscited by 38
  11. 2024Programming Every Example: Lifting Pre-training Data Quality Like Experts at Scalecited by 37
  12. 2024Harnessing Diversity for Important Data Selection in Pretraining Large Language Modelscited by 34
  13. 2024Data curation via joint example selection further accelerates multimodal learningcited by 33
  14. 2025Trajectory Balance with Asynchrony: Decoupling Exploration and Learning for Fast, Scalable LLM Post-Trainingcited by 29
  15. 2025Shifting AI Efficiency From Model-Centric to Data-Centric Compressioncited by 29
  16. 2025SPEED-RL: Faster Training of Reasoning Models via Online Curriculum Learningcited by 29
  17. 2024Efficient Continual Pre-training by Mitigating the Stability Gapcited by 25
  18. 2024Selective Preference Optimization via Token-Level Reward Function Estimationcited by 25
  19. 2024How to Synthesize Text Data without Model Collapse?cited by 25
  20. 2025Learning to Reason at the Frontier of Learnabilitycited by 23
  21. 2024AutoScale: Scale-Aware Data Mixing for Pre-Training LLMscited by 22
  22. 2024High-dimensional Analysis of Knowledge Distillation: Weak-to-Strong Generalization and Scaling Lawscited by 22
  23. 2025Knapsack RL: Unlocking Exploration of LLMs via Optimizing Budget Allocationcited by 22
  24. 2025SelecTKD: Selective Token-Weighted Knowledge Distillation for LLMscited by 22
  25. 2024Seeing the Image: Prioritizing Visual Correlation by Contrastive Alignmentcited by 20
  26. 2024CoLoR-Filter: Conditional Loss Reduction Filtering for Targeted Language Model Pre-trainingcited by 20
  27. 2025Language Models Improve When Pretraining Data Matches Target Taskscited by 19
  28. 2025RefineX: Learning to Refine Pre-training Data at Scale from Expert-Guided Programscited by 17
  29. 2025Think-at-Hard: Dynamic Looped Transformers for Improved Reasoningcited by 16
  30. 2024Understanding the Gains from Repeated Self-Distillationcited by 15
  31. 2024A Little Help Goes a Long Way: Efficient LLM Training by Leveraging Small LMscited by 15
  32. 2025L-MTP: Leap Multi-Token Prediction Beyond Adjacent Context for Large Language Modelscited by 15
  33. 2025One Example Shown, Many Concepts Known! Counterexample-Driven Conceptual Reasoning in Mathematical LLMscited by 14
  34. 2024LLM-RG4: Flexible and Factual Radiology Report Generation across Diverse Input Contextscited by 13
  35. 2025Learning to Focus: Causal Attention Distillation via Gradient-Guided Token Pruningcited by 13
  36. 2025Midtraining Bridges Pretraining and Posttraining Distributionscited by 12
  37. 2026GradAlign: Gradient-Aligned Data Selection for LLM Reinforcement Learningcited by 12
  38. 2026Understanding LoRA as Knowledge Memory: An Empirical Analysiscited by 12

50 most cited of 133 All 133 (JSON)