mlsanityData to 7 Oct 2026

CWM: An Open-Weights LLM for Research on Code Generation with World Models

arXiv 2510.02387 ↗First posted 30 Sep 2025cs.SE cs.AI cs.LGcited by 87

On the map: World models → World models for language agents

Abstract (arXiv)

We release Code World Model (CWM), a 32-billion-parameter open-weights LLM, to advance research on code generation with world models. To improve code understanding beyond what can be learned from training on static code alone, we mid-train CWM on a large amount of observation-action trajectories from Python interpreter and agentic Docker environments, and perform extensive multi-task reasoning RL in verifiable coding, math, and multi-turn software engineering environments. With CWM, we provide a strong testbed for researchers to explore the opportunities world modeling affords for improving code generation with reasoning and planning in computational environments. We present first steps of how world models can benefit agentic coding, enable step-by-step simulation of Python code execution, and show early results of how reasoning can benefit from the latter. CWM is a dense, decoder-only LLM trained with a context size of up to 131k tokens. Independent of its world modeling capabilities, CWM offers strong performance on general coding and math tasks: it reaches pass@1 scores of 65.8% on SWE-bench Verified (with test-time scaling), 68.6% on LiveCodeBench, 96.6% on Math-500, and 76.0% on AIME 2024. To support further research on code world modeling, we release model checkpoints after mid-training, SFT, and RL.

Key facts. Cited by 87 papers in the map (data through 7 Oct 2026). Placed in World models for language agents, one of 11 directions in World models. Builds on 55 of its 115 references that are in the map. First posted on arXiv 30 Sep 2025.

Builds on

55 of 115 references.

References · 55

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  3. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  4. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  5. 2025Qwen3 Technical Reportcited by 8,195
  6. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  7. 2020Scaling Laws for Neural Language Modelscited by 6,411
  8. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  9. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  10. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  11. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  12. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
38 more
  1. 2021Program Synthesis with Large Language Modelscited by 3,509
  2. 2022Training Compute-Optimal Large Language Modelscited by 2,927
  3. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
  4. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,678
  5. 2023Code Llama: Open Foundation Models for Codecited by 2,105
  6. 2020GLU Variants Improve Transformercited by 1,724
  7. 2022Competition-Level Code Generation with AlphaCodecited by 1,427
  8. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,311
  9. 2025Understanding R1-Zero-Like Training: A Critical Perspectivecited by 1,276
  10. 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,270
  11. 2025gpt-oss-120b & gpt-oss-20b Model Cardcited by 1,258
  12. 2021Measuring Coding Challenge Competence With APPScited by 879
  13. 2024DeepSeek LLM: Scaling Open-Source Language Models with Longtermismcited by 621
  14. 2025Kimi K2: Open Agentic Intelligencecited by 523
  15. 2024From Crowdsourced Data to High-Quality Benchmarks: Arena-Hard and BenchBuilder Pipelinecited by 484
  16. 2025Open-Reasoner-Zero: An Open Source Approach to Scaling Up Reinforcement Learning on the Base Modelcited by 450
  17. 2021MiniF2F: a cross-system benchmark for formal Olympiad-level mathematicscited by 380
  18. 2024Measuring short-form factuality in large language modelscited by 345
  19. 2023Effective Long-Context Scaling of Foundation Modelscited by 301
  20. 2024Training Software Engineering Agents and Verifiers with SWE-Gymcited by 300
  21. 2024CRUXEval: A Benchmark for Code Reasoning, Understanding and Executioncited by 294
  22. 2025SWE-smith: Scaling Data for Software Engineering Agentscited by 255
  23. 2025SWE-RL: Advancing LLM Reasoning via Reinforcement Learning on Open Software Evolutioncited by 246
  24. 2022FP8 Formats for Deep Learningcited by 187
  25. 2023ProofNet: Autoformalizing and Formally Proving Undergraduate-Level Mathematicscited by 180
  26. 2025AIMO-2 Winning Solution: Building State-of-the-Art Mathematical Reasoning Models with OpenMathReasoning datasetcited by 165
  27. 2024RLEF: Grounding Code LLMs in Execution Feedback with Reinforcement Learningcited by 146
  28. 2025R2E-Gym: Procedural Environments and Hybrid Verifiers for Scaling Open-Weights SWE Agentscited by 140
  29. 2025Goedel-Prover: A Frontier Model for Open-Source Automated Theorem Provingcited by 126
  30. 2024Lean Workbook: A large-scale Lean problem set formalized from natural language math problemscited by 115
  31. 2025OpenCodeReasoning: Advancing Data Distillation for Competitive Codingcited by 106
  32. 2024Language models scale reliably with over-training and on downstream taskscited by 105
  33. 2025Which Economic Tasks are Performed with AI? Evidence from Millions of Claude Conversationscited by 88
  34. 2025Don't Overthink it. Preferring Shorter Thinking Chains for Improved LLM Reasoningcited by 82
  35. 2025Magistralcited by 57
  36. 2025Optimizing Language Models for Inference Time Objectives using Reinforcement Learningcited by 49
  37. 2024Meta Large Language Model Compiler: Foundation Models of Compiler Optimizationcited by 39
  38. 2024HARP: A challenging human-annotated math reasoning benchmarkcited by 25

50 most cited of 55 All 55 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 14; 2026: 73.

87 papers.

Sort:

By area:

  1. 2025The Art of Scaling Reinforcement Learning Compute for LLMscited by 91
  2. 2025Live-SWE-agent: Can Software Engineering Agents Self-Evolve on the Fly?cited by 91
  3. 2025Agent Learning via Early Experiencecited by 79
  4. 2026Qwen3-Coder-Next Technical Reportcited by 60
  5. 2025SWE-EVO: Benchmarking Coding Agents in Long-Horizon Software Evolution Scenarioscited by 49
  6. 2026Expanding the Capabilities of Reinforcement Learning via Text Feedbackcited by 49
  7. 2026A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agentscited by 47
  8. 2026Endless Terminals: Scaling RL Environments for Terminal Agentscited by 43
  9. 2026Code as Agent Harnesscited by 40
  10. 2025Toward Training Superintelligent Software Agents through Self-Play SWE-RLcited by 37
  11. 2026AIRS-Bench: a Suite of Tasks for Frontier AI Research Science Agentscited by 32
  12. 2026SWE-Lego: Pushing the Limits of Supervised Fine-tuning for Software Issue Resolvingcited by 30
38 more
  1. 2026Qwen-AgentWorld: Language World Models for General Agentscited by 22
  2. 2026SWE-World: Building Software Engineering Agents in Docker-Free Environmentscited by 21
  3. 2026Visual Generation Unlocks Human-Like Reasoning through Multimodal World Modelscited by 18
  4. 2026SWE-Master: Unleashing the Potential of Software Engineering Agents via Post-Trainingcited by 18
  5. 2026SWE-Universe: Scale Real-World Verifiable Environments to Millionscited by 16
  6. 2026Scaling the Horizon, Not the Parameters: Reaching Trillion-Parameter Performance with a 35B Agentcited by 16
  7. 2026Current Agents Fail to Leverage World Model as Tool for Foresightcited by 15
  8. 2025From Code Foundation Models to Agents and Applications: A Comprehensive Survey and Practical Guide to Code Intelligencecited by 13
  9. 2026Generative Visual Code Mobile World Modelscited by 13
  10. 2026SWE-Replay: Efficient Test-Time Scaling for Software Engineering Agentscited by 11
  11. 2026Can We Predict Before Executing Machine Learning Agents?cited by 10
  12. 2026SERA: Soft-Verified Efficient Repository Agentscited by 10
  13. 2026Agentic World Modeling: Foundations, Capabilities, Laws, and Beyondcited by 10
  14. 2026Code World Model: Coding Agent as World Braincited by 9
  15. 2025Youtu-LLM: Unlocking the Native Agentic Potential for Lightweight Large Language Modelscited by 8
  16. 2026Agentic Environment Engineering for Large Language Models: A Survey of Environment Modeling, Synthesis, Evaluation, and Applicationcited by 8
  17. 2026AI Research Preference Modelscited by 7
  18. 2025SAGE: Training Smart Any-Horizon Agents for Long Video Reasoning with Reinforcement Learningcited by 6
  19. 2025A Unified Definition of Hallucination: It's The World Model, Stupid!cited by 6
  20. 2025Agentic Software Issue Resolution with Large Language Models: A Surveycited by 6
  21. 2025LLMs Lean on Priors, Not Programming Language Semanticscited by 5
  22. 2026Aligning Agentic World Models via Knowledgeable Experience Learningcited by 5
  23. 2026Self-Improving World Modelling with Latent Actionscited by 5
  24. 2026Learning State-Tracking from Code Using Linear RNNscited by 5
  25. 2026Self-Improvement of Large Language Models: A Technical Overview and Future Outlookcited by 5
  26. 2026From SWE-ZERO to SWE-HERO: Execution-free to Execution-based Fine-tuning for Software Engineering Agentscited by 5
  27. 2026PatchWorld: Gradient-Free Optimization of Executable World Models for Agent Environmentscited by 5
  28. 2026Aletheia: What Makes RLVR For Code Verifiers Tick?cited by 4
  29. 2026Debugging code world modelscited by 4
  30. 2026InCoder-32B-Thinking: Industrial Code World Model for Thinkingcited by 4
  31. 2026ZAYA1-8B Technical Reportcited by 4
  32. 2025Evaluating Code Reasoning Abilities of Large Language Models Under Real-World Settingscited by 3
  33. 2025KernelEvolve: Scaling Agentic Kernel Coding for Heterogeneous AI Accelerators at Metacited by 3
  34. 2026SWE-Prot'eg'e: Learning to Selectively Collaborate With an Expert Unlocks Small Language Models as Software Engineering Agentscited by 3
  35. 2026A Deep Dive into Scaling RL for Code Generation with Synthetic Data and Curriculacited by 3
  36. 2026OpenWorldLib: A Unified Codebase and Definition of Advanced World Modelscited by 3
  37. 2026Beyond RAG for Cyber Threat Intelligence: A Systematic Evaluation of Graph-Based and Agentic Retrievalcited by 3
  38. 2026Agentic Discovery of Neural Architectures: AIRA-Compose and AIRA-Designcited by 3

50 most cited of 87 All 87 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Generating Code World Models with Large Language Models Guided by Monte Carlo Tree Search
  2. 2026CodeEval: A pedagogical approach for targeted evaluation of code-trained Large Language Models
  3. 2024WorldCoder, a Model-Based LLM Agent: Building World Models by Writing Code and Interacting with the Environment
  4. 2026Evaluating LLMs Code Reasoning Under Real-World Context
  5. 2026Distilling Game Code World Model Generation into Lightweight Large Language Models
  6. 2026FormulaCode: Evaluating Agentic Optimization on Large Codebases
  7. 2024Granite Code Models: A Family of Open Foundation Models for Code Intelligence
  8. 2025Enhancing LLM Code Generation: A Systematic Evaluation of Multi-Agent Collaboration and Runtime Debugging for Improved Accuracy, Reliability, and Latency