mlsanityData to 7 Oct 2026

2 OLMo 2 Furious

arXiv 2501.00656 ↗First posted 31 Dec 2024cs.CL cs.LGcited by 302

Not on the map. Why?

Cited by 68 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We present OLMo 2, the next generation of our fully open language models. OLMo 2 includes a family of dense autoregressive language models at 7B, 13B and 32B scales with fully released artifacts -- model weights, full training data, training code and recipes, training logs and thousands of intermediate checkpoints. In this work, we describe our modified model architecture and training recipe, focusing on techniques for achieving better training stability and improved per-token efficiency. Our updated pretraining data mixture introduces a new, specialized data mix called Dolmino Mix 1124, which significantly improves model capabilities across many downstream task benchmarks when introduced via late-stage curriculum training (i.e. specialized data during the annealing phase of pretraining). Finally, we incorporate best practices from T"ulu 3 to develop OLMo 2-Instruct, focusing on permissive data and extending our final-stage reinforcement learning with verifiable rewards (RLVR). Our OLMo 2 base models sit at the Pareto frontier of performance to training compute, often matching or outperforming open-weight only models like Llama 3.1, Qwen 2.5, and Gemma 2 while using fewer FLOPs and with fully transparent training data, code, and recipe. Our fully open OLMo 2-Instruct models are competitive with open-weight only models of comparable size and even some proprietary models like GPT-3.5 Turbo and GPT 4o Mini.

Key facts. Cited by 302 papers in the map (data through 7 Oct 2026). First posted on arXiv 31 Dec 2024.

Builds on

0 of 0 references.

No references here yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 170; 2026: 132.

302 papers.

Sort:

By area:

  1. 2025LLMs Get Lost In Multi-Turn Conversationcited by 400
  2. 2025Muon is Scalable for LLM Trainingcited by 381
  3. 2025Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approachcited by 372
  4. 2025Skywork-Reward-V2: Scaling Preference Data Curation via Human-AI Synergycited by 204
  5. 2025Learning to Reason without External Rewardscited by 199
  6. 2025Spurious Rewards: Rethinking Training Signals in RLVRcited by 199
  7. 2025Generalizing Verifiable Instruction Followingcited by 178
  8. 2025A Survey of Reinforcement Learning for Large Reasoning Modelscited by 175
  9. 2025Echo Chamber: RL Post-training Amplifies Behaviors Learned in Pretrainingcited by 123
  10. 2025olmOCR: Unlocking Trillions of Tokens in PDFs with Vision Language Modelscited by 120
  11. 2025Checklists Are Better Than Reward Models For Aligning Language Modelscited by 115
  12. 2025OctoThinker: Mid-training Incentivizes Reinforcement Learning Scalingcited by 110
38 more
  1. 2025Beyond Binary Rewards: Training LMs to Reason About Their Uncertaintycited by 107
  2. 2025Reasoning or Memorization? Unreliable Results of Reinforcement Learning Due to Data Contaminationcited by 81
  3. 2025Organize the Web: Constructing Domains Enhances Pre-Training Data Curationcited by 73
  4. 2025Don't be lazy: CompleteP enables compute-efficient deep transformerscited by 69
  5. 2025MultiBLiMP 1.0: A Massively Multilingual Benchmark of Linguistic Minimal Pairscited by 65
  6. 2025NoveltyBench: Evaluating Language Models for Humanlike Diversitycited by 64
  7. 2025Utility Engineering: Analyzing and Controlling Emergent Value Systems in AIscited by 63
  8. 2025CUDA-L1: Improving CUDA Optimization via Contrastive Reinforcement Learningcited by 59
  9. 2025LongCat-Flash Technical Reportcited by 54
  10. 2025Reinforcement Learning from Human Feedbackcited by 53
  11. 2025Do Language Models Use Their Depth Efficiently?cited by 48
  12. 2025Beyond Chemical QA: Evaluating LLM's Chemical Reasoning with Modular Chemical Operationscited by 46
  13. 2025Rethinking Reflection in Pre-Trainingcited by 43
  14. 2025Teaching Pretrained Language Models to Think Deeper with Retrofitted Recurrencecited by 41
  15. 2025The Avengers: A Simple Recipe for Uniting Smaller Language Models to Challenge Proprietary Giantscited by 37
  16. 2025LLM-JEPA: Large Language Models Meet Joint Embedding Predictive Architecturescited by 35
  17. 2025Linguistic Generalizability of Test-Time Scaling in Mathematical Reasoningcited by 33
  18. 2025Tracing the Representation Geometry of Language Models from Pretraining to Post-trainingcited by 33
  19. 2025Peri-LN: Revisiting Normalization Layer in the Transformer Architecturecited by 32
  20. 2025Fluid Language Model Benchmarkingcited by 30
  21. 2025Language Models Fail to Introspect About Their Knowledge of Languagecited by 28
  22. 2025LLM Probability Concentration: How Alignment Shrinks the Generative Horizoncited by 26
  23. 2025EXAONE 4.0: Unified Large Language Models Integrating Non-reasoning and Reasoning Modescited by 26
  24. 2025Restoring Calibration for Aligned Large Language Models: A Calibration-Aware Fine-Tuning Approachcited by 25
  25. 2025The Delta Learning Hypothesis: Preference Tuning on Weak Data can Yield Strong Gainscited by 25
  26. 2025Encode, Think, Decode: Scaling test-time reasoning with recursive latent thoughtscited by 23
  27. 2025Extending the Context of Pretrained LLMs by Dropping Their Positional Embeddingscited by 23
  28. 2026Purified OPSD: On-Policy Self-Distillation Without Losing How to Thinkcited by 23
  29. 2025IssueBench: Millions of Realistic Prompts for Measuring Issue Bias in LLM Writing Assistancecited by 22
  30. 2025A Multi-Power Law for Loss Curve Prediction Across Learning Rate Schedulescited by 22
  31. 2025Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluationcited by 22
  32. 2025The Reasoning-Memorization Interplay in Language Models Is Mediated by a Single Directioncited by 21
  33. 2025Llama-3.1-FoundationAI-SecurityLLM-8B-Instruct Technical Reportcited by 21
  34. 2025Enhancing Large Language Model Reasoning via Selective Critical Token Fine-Tuningcited by 20
  35. 2025Gemstones: A Model Suite for Multi-Faceted Scaling Lawscited by 19
  36. 2025BeyondWeb: Lessons from Scaling Synthetic Data for Trillion-scale Pretrainingcited by 19
  37. 2025Mid-Training of Large Language Models: A Surveycited by 18
  38. 2025BLEUBERI: BLEU is a surprisingly effective reward for instruction followingcited by 17

50 most cited of 302 All 302 (JSON)