mlsanityData to 7 Oct 2026

Qwen2.5-Coder Technical Report

arXiv 2409.12186 ↗First posted 18 Sep 2024cs.CLcited by 1,151

On the map: LLM reasoning → Reasoning for agents, planning and code

Abstract (arXiv)

In this report, we introduce the Qwen2.5-Coder series, a significant upgrade from its predecessor, CodeQwen1.5. This series includes six models: Qwen2.5-Coder-(0.5B/1.5B/3B/7B/14B/32B). As a code-specific model, Qwen2.5-Coder is built upon the Qwen2.5 architecture and continues pretrained on a vast corpus of over 5.5 trillion tokens. Through meticulous data cleaning, scalable synthetic data generation, and balanced data mixing, Qwen2.5-Coder demonstrates impressive code generation capabilities while retaining general and math skills. These models have been evaluated on a wide range of code-related tasks, achieving state-of-the-art (SOTA) performance across more than 10 benchmarks, including code generation, completion, reasoning, and repair, consistently outperforming larger models of the same model size. We believe that the release of the Qwen2.5-Coder series will advance research in code intelligence and, with its permissive licensing, support wider adoption by developers in real-world applications.

Key facts. Cited by 1,151 papers in the map (data through 7 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 46 of its 53 references that are in the map. First posted on arXiv 18 Sep 2024.

Builds on

46 of 53 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code12 Mar

Other references · 45

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2024The Llama 3 Herd of Modelscited by 14,192
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  5. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  6. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  7. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  8. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  9. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  10. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  11. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  12. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
33 more
  1. 2021Program Synthesis with Large Language Modelscited by 3,509
  2. 2023Qwen Technical Reportcited by 3,273
  3. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  4. 2023Code Llama: Open Foundation Models for Codecited by 2,105
  5. 2024Qwen2 Technical Reportcited by 2,076
  6. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,332
  7. 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,286
  8. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,154
  9. 2024Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preferencecited by 1,151
  10. 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,120
  11. 2023StarCoder: may the source be with you!cited by 790
  12. 2023Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLscited by 667
  13. 2023YaRN: Efficient Context Window Extension of Large Language Modelscited by 584
  14. 2024BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructionscited by 527
  15. 2024StarCoder 2 and The Stack v2: The Next Generationcited by 484
  16. 2023RepoCoder: Repository-Level Code Completion Through Iterative Retrieval and Generationcited by 357
  17. 2024CRUXEval: A Benchmark for Code Reasoning, Understanding and Executioncited by 294
  18. 2023Magicoder: Empowering Code Generation with OSS-Instructcited by 251
  19. 2022Efficient Training of Language Models to Fill in the Middlecited by 248
  20. 2023TheoremQA: A Theorem-driven Question Answering datasetcited by 189
  21. 2023CrossCodeEval: A Diverse and Multilingual Benchmark for Cross-File Code Completioncited by 181
  22. 2024Are We Done with MMLU?cited by 178
  23. 2023SantaCoder: don't reach for the stars!cited by 169
  24. 2022MultiPL-E: A Scalable and Extensible Approach to Benchmarking Neural Code Generationcited by 132
  25. 2024TableBench: A Comprehensive and Complex Benchmark for Table Question Answeringcited by 127
  26. 2022ReACC: A Retrieval-Augmented Code Completion Frameworkcited by 117
  27. 2023How to Prompt LLMs for Text-to-SQL: A Study in Zero-shot, Single-domain, and Cross-domain Settingscited by 57
  28. 2024Repoformer: Selective Retrieval for Repository-Level Code Completioncited by 55
  29. 2023WaveCoder: Widespread And Versatile Enhancement For Code Large Language Models By Instruction Tuningcited by 42
  30. 2024McEval: Massively Multilingual Code Evaluationcited by 42
  31. 2024Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Taskscited by 24
  32. 2024CodeEditorBench: Evaluating Code Editing Capability of Large Language Modelscited by 22
  33. 2024UniCoder: Scaling Code Large Language Model via Universal Codecited by 18

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 80; 2025: 600; 2026: 471.

1,151 papers.

Sort:

By area:

  1. 2025Qwen3 Technical Reportcited by 8,195
  2. 2024Qwen2.5 Technical Reportcited by 4,713
  3. 2025Why Do Multi-Agent LLM Systems Fail?cited by 559
  4. 2025Visual-RFT: Visual Reinforcement Fine-Tuningcited by 501
  5. 2025Open-Reasoner-Zero: An Open Source Approach to Scaling Up Reinforcement Learning on the Base Modelcited by 450
  6. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 381
  7. 2025Absolute Zero: Reinforced Self-play Reasoning with Zero Datacited by 323
  8. 2025WebThinker: Empowering Large Reasoning Models with Deep Research Capabilitycited by 320
  9. 2024Training Software Engineering Agents and Verifiers with SWE-Gymcited by 300
  10. 2024ProcessBench: Identifying Process Errors in Mathematical Reasoningcited by 277
  11. 2025SWE-smith: Scaling Data for Software Engineering Agentscited by 255
  12. 2025AReaL: A Large-Scale Asynchronous Reinforcement Learning System for Language Reasoningcited by 234
38 more
  1. 2025Mercury: Ultra-Fast Language Models Based on Diffusioncited by 225
  2. 2026Self-Distillation Enables Continual Learningcited by 212
  3. 2025Skywork-Reward-V2: Scaling Preference Data Curation via Human-AI Synergycited by 204
  4. 2025DiffuCoder: Understanding and Improving Masked Diffusion Models for Code Generationcited by 184
  5. 2025Scalable Best-of-N Selection for Large Language Models via Self-Certaintycited by 182
  6. 2025Qwen2.5-1M Technical Reportcited by 158
  7. 2025LLM Post-Training: A Deep Dive into Reasoning Large Language Modelscited by 129
  8. 2024OpenCoder: The Open Cookbook for Top-Tier Code Large Language Modelscited by 127
  9. 2025Qwen3Guard Technical Reportcited by 125
  10. 2025Process Reward Models That Thinkcited by 118
  11. 2025KodCode: A Diverse, Challenging, and Verifiable Synthetic Dataset for Codingcited by 115
  12. 2025ELEPHANT: Measuring and understanding social sycophancy in LLMscited by 115
  13. 2025CodeElo: Benchmarking Competition-level Code Generation of LLMs with Human-comparable Elo Ratingscited by 109
  14. 2025A Survey of Frontiers in LLM Reasoning: Inference Scaling, Learning to Reason, and Agentic Systemscited by 108
  15. 2025OpenCodeReasoning: Advancing Data Distillation for Competitive Codingcited by 106
  16. 2025S*: Test Time Scaling for Code Generationcited by 95
  17. 2025Audio-Reasoner: Improving Reasoning Capability in Large Audio Language Modelscited by 94
  18. 2025Q-Insight: Understanding Image Quality via Visual Reinforcement Learningcited by 93
  19. 2025GuardReasoner: Towards Reasoning-based LLM Safeguardscited by 89
  20. 2025On-Policy RL Meets Off-Policy Experts: Harmonizing Supervised Fine-Tuning and Reinforcement Learning via Dynamic Weightingcited by 88
  21. 2025Unified Multimodal Chain-of-Thought Reward Model through Reinforcement Fine-Tuningcited by 86
  22. 2025UniVG-R1: Reasoning Guided Universal Visual Grounding with Reinforcement Learningcited by 83
  23. 2024Enhancing Code LLMs with Reinforcement Learning in Code Generation: A Surveycited by 82
  24. 2025Dream-Coder 7B: An Open Diffusion Language Model for Codecited by 81
  25. 2025ReMA: Learning to Meta-think for LLMs with Multi-Agent Reinforcement Learningcited by 78
  26. 2024AceMath: Advancing Frontier Math Reasoning with Post-Training and Reward Modelingcited by 76
  27. 2025AceReason-Nemotron 1.1: Advancing Math and Code Reasoning through SFT and RL Synergycited by 75
  28. 2025Chain-of-Agents: End-to-End Agent Foundation Models via Multi-Agent Distillation and Agentic RLcited by 75
  29. 2024FlipAttack: Jailbreak LLMs via Flippingcited by 73
  30. 2025DistiLLM-2: A Contrastive Approach Boosts the Distillation of LLMscited by 73
  31. 2026CaP-X: A Framework for Benchmarking and Improving Coding Agents for Robot Manipulationcited by 72
  32. 2024A Unified Approach to Routing and Cascading for LLMscited by 71
  33. 2025UserBench: An Interactive Gym Environment for User-Centric Agentscited by 69
  34. 2025LeetCodeDataset: A Temporal Dataset for Robust Evaluation and Efficient Training of Code LLMscited by 67
  35. 2025Paper2Code: Automating Code Generation from Scientific Papers in Machine Learningcited by 67
  36. 2025WebGen-Bench: Evaluating LLMs on Generating Interactive and Functional Websites from Scratchcited by 66
  37. 2025Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineeringcited by 64
  38. 2024Liquid: Language Models are Scalable and Unified Multi-modal Generatorscited by 62

50 most cited of 1,151 All 1,151 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvement
  2. 2026IQuest-Coder-V1 Technical Report
  3. 2025Qwen3-VL Technical Report
  4. 2024MathCoder2: Better Math Reasoning from Continued Pretraining on Model-translated Mathematical Code
  5. 2026CodeEval: A pedagogical approach for targeted evaluation of code-trained Large Language Models
  6. 2024CodeAgent: Enhancing Code Generation with Tool-Integrated Agent Systems for Real-World Repo-level Coding Challenges
  7. 2024Benchmarking Large Language Models with Integer Sequence Generation Tasks
  8. 2025Increasing LLM Coding Capabilities through Diverse Synthetic Coding Tasks