mlsanityData to 7 Oct 2026

DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligence

arXiv 2406.11931 ↗First posted 17 Jun 2024cs.SE cs.AI cs.LGcited by 325

On the map: Mixture-of-experts → MoE large language models

Abstract (arXiv)

We present DeepSeek-Coder-V2, an open-source Mixture-of-Experts (MoE) code language model that achieves performance comparable to GPT4-Turbo in code-specific tasks. Specifically, DeepSeek-Coder-V2 is further pre-trained from an intermediate checkpoint of DeepSeek-V2 with additional 6 trillion tokens. Through this continued pre-training, DeepSeek-Coder-V2 substantially enhances the coding and mathematical reasoning capabilities of DeepSeek-V2, while maintaining comparable performance in general language tasks. Compared to DeepSeek-Coder-33B, DeepSeek-Coder-V2 demonstrates significant advancements in various aspects of code-related tasks, as well as reasoning and general capabilities. Additionally, DeepSeek-Coder-V2 expands its support for programming languages from 86 to 338, while extending the context length from 16K to 128K. In standard benchmark evaluations, DeepSeek-Coder-V2 achieves superior performance compared to closed-source models such as GPT4-Turbo, Claude 3 Opus, and Gemini 1.5 Pro in coding and math benchmarks.

Key facts. Cited by 325 papers in the map (data through 7 Oct 2026). Placed in MoE large language models, one of 12 directions in Mixture-of-experts. Builds on 31 of its 41 references that are in the map. First posted on arXiv 17 Jun 2024.

Builds on

31 of 41 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model7 May

Other references · 30

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  4. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  5. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  6. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  7. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  8. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  9. 2021Program Synthesis with Large Language Modelscited by 3,509
  10. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
  11. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
  12. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
18 more
  1. 2023Code Llama: Open Foundation Models for Codecited by 2,105
  2. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,767
  3. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,332
  4. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,154
  5. 2024Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluatorscited by 865
  6. 2023AGIEval: A Human-Centric Benchmark for Evaluating Foundation Modelscited by 701
  7. 2023C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Modelscited by 694
  8. 2023YaRN: Efficient Context Window Extension of Large Language Modelscited by 584
  9. 2024StarCoder 2 and The Stack v2: The Next Generationcited by 484
  10. 2023CMMLU: Measuring massive multitask language understanding in Chinesecited by 436
  11. 2022The Stack: 3 TB of permissively licensed source codecited by 354
  12. 2023RepoBench: Benchmarking Repository-Level Code Auto-Completion Systemscited by 346
  13. 2016FastText.zip: Compressing text classification modelscited by 331
  14. 2020CLUE: A Chinese Language Understanding Evaluation Benchmarkcited by 277
  15. 2022Efficient Training of Language Models to Fill in the Middlecited by 248
  16. 2023SantaCoder: don't reach for the stars!cited by 169
  17. 2024Can Language Models Solve Olympiad Programming?cited by 70
  18. 2023AlignBench: Benchmarking Chinese Alignment of Large Language Modelscited by 68

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 74; 2025: 180; 2026: 71.

325 papers.

Sort:

By area:

  1. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
  2. 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,057
  3. 2024DeepSeek-V3 Technical Reportcited by 724
  4. 2025The Lessons of Developing Process Reward Models in Mathematical Reasoningcited by 402
  5. 2024Scaling Synthetic Data Creation with 1,000,000,000 Personascited by 397
  6. 2025RAGEN: Understanding Self-Evolution in LLM Agents via Multi-Turn Reinforcement Learningcited by 357
  7. 2024Step-DPO: Step-wise Preference Optimization for Long-chain Reasoning of LLMscited by 265
  8. 2025Mercury: Ultra-Fast Language Models Based on Diffusioncited by 225
  9. 2024Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflowscited by 223
  10. 2024OpenMathInstruct-2: Accelerating AI for Math with Massive Open-Source Instruction Datacited by 194
  11. 2024Auxiliary-Loss-Free Load Balancing Strategy for Mixture-of-Expertscited by 191
  12. 2024DeepSeek-Prover-V1.5: Harnessing Proof Assistant Feedback for Reinforcement Learning and Monte-Carlo Tree Searchcited by 189
38 more
  1. 2024SciCode: A Research Coding Benchmark Curated by Scientistscited by 172
  2. 2024RLEF: Grounding Code LLMs in Execution Feedback with Reinforcement Learningcited by 146
  3. 2025Can We Generate Images with CoT? Let's Verify and Reinforce Image Generation Step by Stepcited by 130
  4. 2024OpenCoder: The Open Cookbook for Top-Tier Code Large Language Modelscited by 127
  5. 2025MiMo: Unlocking the Reasoning Potential of Language Model -- From Pretraining to Posttrainingcited by 117
  6. 2025CodeElo: Benchmarking Competition-level Code Generation of LLMs with Human-comparable Elo Ratingscited by 109
  7. 2025A Survey of Frontiers in LLM Reasoning: Inference Scaling, Learning to Reason, and Agentic Systemscited by 108
  8. 2024LLaMA-Berry: Pairwise Optimization for O1-like Olympiad-Level Mathematical Reasoningcited by 90
  9. 2025OmniSQL: Synthesizing High-quality Text-to-SQL Data at Scalecited by 90
  10. 2025SPIRAL: Self-Play on Zero-Sum Games Incentivizes Reasoning via Multi-Agent Multi-Turn Reinforcement Learningcited by 89
  11. 2025T1: Advancing Language Model Reasoning through Reinforcement Learning and Inference Scalingcited by 77
  12. 2025TreeRL: LLM Reinforcement Learning with On-Policy Tree Searchcited by 75
  13. 2024MarsCode Agent: AI-native Automated Bug Fixingcited by 68
  14. 2025Paper2Code: Automating Code Generation from Scientific Papers in Machine Learningcited by 67
  15. 2025ACEBench: Who Wins the Match Point in Tool Usage?cited by 66
  16. 2025Can LLMs Replace Human Evaluators? An Empirical Study of LLM-as-a-Judge in Software Engineeringcited by 64
  17. 2024CodexGraph: Bridging Large Language Models and Code Repositories via Code Graph Databasescited by 62
  18. 2024DA-Code: Agent Data Science Code Generation Benchmark for Large Language Modelscited by 57
  19. 2024Lingma SWE-GPT: An Open Development-Process-Centric Language Model for Automated Software Improvementcited by 57
  20. 2025FEA-Bench: A Benchmark for Evaluating Repository-Level Code Generation for Feature Implementationcited by 57
  21. 2025VideoRAG: Retrieval-Augmented Generation over Video Corpuscited by 55
  22. 2025CodeI/O: Condensing Reasoning Patterns via Code Input-Output Predictioncited by 54
  23. 2024ChatQA 2: Bridging the Gap to Proprietary LLMs in Long Context and RAG Capabilitiescited by 52
  24. 2025Rethinking Mixture-of-Agents: Is Mixing Different Large Language Models Beneficial?cited by 52
  25. 2025Towards Robust Agentic CUDA Kernel Benchmarking, Verification, and Optimizationcited by 52
  26. 2024GraphArena: Evaluating and Exploring Large Language Models on Graph Computationcited by 48
  27. 2024To Code, or Not To Code? Exploring Impact of Code in Pre-trainingcited by 47
  28. 2025Code to Think, Think to Code: A Survey on Code-Enhanced Reasoning and Reasoning-Driven Code Intelligence in LLMscited by 47
  29. 2024From Code to Correctness: Closing the Last Mile of Code Generation with Hierarchical Debuggingcited by 46
  30. 2025Seed-Coder: Let the Code Model Curate Data for Itselfcited by 46
  31. 2025Feedback Descent: Open-Ended Text Optimization via Pairwise Comparisoncited by 42
  32. 2024Improving LLM Reasoning through Scaling Inference Computation with Collaborative Verificationcited by 40
  33. 2024HumanEval Pro and MBPP Pro: Evaluating Large Language Models on Self-invoking Code Generationcited by 39
  34. 2025Insights into DeepSeek-V3: Scaling Challenges and Reflections on Hardware for AI Architecturescited by 38
  35. 2025Arctic-Text2SQL-R1: Simple Rewards, Strong Reasoning in Text-to-SQLcited by 38
  36. 2025SWE-QA: Can Language Models Answer Repository-level Code Questions?cited by 37
  37. 2025OmniRouter: Budget and Performance Controllable Multi-LLM Routingcited by 35
  38. 2025Solving Formal Math Problems by Decomposition and Iterative Reflectioncited by 35

50 most cited of 325 All 325 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024DeepSeek LLM: Scaling Open-Source Language Models with Longtermism
  2. 2025Quantifying the Capability Boundary of DeepSeek Models: An Application-Driven Performance Analysis
  3. 2025DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models
  4. 2024Granite Code Models: A Family of Open Foundation Models for Code Intelligence
  5. 2025FLAME-MoE: A Transparent End-to-End Research Platform for Mixture-of-Experts Language Models
  6. 2026DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence
  7. 2024DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understanding
  8. 2025DeepSeek-Prover-V2: Advancing Formal Mathematical Reasoning via Reinforcement Learning for Subgoal Decomposition