mlsanityData to 7 Oct 2026

StarCoder 2 and The Stack v2: The Next Generation

arXiv 2402.19173 ↗First posted 29 Feb 2024cs.SE cs.AIcited by 484

Not on the map. Why?

Cited by 60 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

The BigCode project, an open-scientific collaboration focused on the responsible development of Large Language Models for Code (Code LLMs), introduces StarCoder2. In partnership with Software Heritage (SWH), we build The Stack v2 on top of the digital commons of their source code archive. Alongside the SWH repositories spanning 619 programming languages, we carefully select other high-quality data sources, such as GitHub pull requests, Kaggle notebooks, and code documentation. This results in a training set that is 4x larger than the first StarCoder dataset. We train StarCoder2 models with 3B, 7B, and 15B parameters on 3.3 to 4.3 trillion tokens and thoroughly evaluate them on a comprehensive set of Code LLM benchmarks. We find that our small model, StarCoder2-3B, outperforms other Code LLMs of similar size on most benchmarks, and also outperforms StarCoderBase-15B. Our large model, StarCoder2- 15B, significantly outperforms other models of comparable size. In addition, it matches or outperforms CodeLlama-34B, a model more than twice its size. Although DeepSeekCoder- 33B is the best-performing model at code completion for high-resource languages, we find that StarCoder2-15B outperforms it on math and code reasoning benchmarks, as well as several low-resource languages. We make the model weights available under an OpenRAIL license and ensure full transparency regarding the training data by releasing the SoftWare Heritage persistent IDentifiers (SWHIDs) of the source code data.

Key facts. Cited by 484 papers in the map (data through 7 Oct 2026). First posted on arXiv 29 Feb 2024.

Builds on

0 of 0 references.

No references here yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 164; 2025: 216; 2026: 104.

484 papers.

Sort:

By area:

  1. 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,442
  2. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,193
  3. 2024Qwen2.5-Coder Technical Reportcited by 1,151
  4. 2024A Survey on Large Language Models for Code Generationcited by 646
  5. 2024BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructionscited by 527
  6. 2024Training Language Models to Self-Correct via Reinforcement Learningcited by 405
  7. 2025Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approachcited by 372
  8. 2024DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligencecited by 325
  9. 2024LoRA Learns Less and Forgets Lesscited by 313
  10. 2024OLMoE: Open Mixture-of-Experts Language Modelscited by 291
  11. 2024InternLM-XComposer-2.5: A Versatile Large Vision Language Model Supporting Long-Contextual Input and Outputcited by 195
  12. 2025DiffuCoder: Understanding and Improving Masked Diffusion Models for Code Generationcited by 184
38 more
  1. 2024Large Language Model-Based Agents for Software Engineering: A Surveycited by 173
  2. 2024Building and better understanding vision-language models: insights and future directionscited by 169
  3. 2024CodeGemma: Open Code Models Based on Gemmacited by 145
  4. 2024Observational Scaling Laws and the Predictability of Language Model Performancecited by 128
  5. 2024OpenCoder: The Open Cookbook for Top-Tier Code Large Language Modelscited by 127
  6. 2024Next-Generation Database Interfaces: A Survey of LLM-based Text-to-SQLcited by 118
  7. 2024Granite Code Models: A Family of Open Foundation Models for Code Intelligencecited by 116
  8. 2024Scaling Laws with Vocabulary: Larger Models Deserve Larger Vocabulariescited by 113
  9. 2024LLM Hallucinations in Practical Code Generation: Phenomena, Mechanism, and Mitigationcited by 92
  10. 2024Scaling Laws for Precisioncited by 92
  11. 2025OmniSQL: Synthesizing High-quality Text-to-SQL Data at Scalecited by 90
  12. 2024CodeRAG-Bench: Can Retrieval Augment Code Generation?cited by 89
  13. 2025ACECODER: Acing Coder RL via Automated Test-Case Synthesiscited by 89
  14. 2025T1: Advancing Language Model Reasoning through Reinforcement Learning and Inference Scalingcited by 77
  15. 2025TreeRL: LLM Reinforcement Learning with On-Policy Tree Searchcited by 75
  16. 2025Practical Efficiency of Muon for Pretrainingcited by 74
  17. 2024EvoCodeBench: An Evolving Code Generation Benchmark Aligned with Real-World Code Repositoriescited by 71
  18. 2024DevEval: A Manually-Annotated Code Generation Benchmark Aligned with Real-World Code Repositoriescited by 71
  19. 2024When LLMs Meet Cybersecurity: A Systematic Literature Reviewcited by 70
  20. 2024MarsCode Agent: AI-native Automated Bug Fixingcited by 68
  21. 2024HyperAgent: Generalist Software Engineering Agents to Solve Coding Tasks at Scalecited by 61
  22. 2024SelfCodeAlign: Self-Alignment for Code Generationcited by 61
  23. 2026Qwen3-Coder-Next Technical Reportcited by 60
  24. 2024Alibaba LingmaAgent: Improving Automated Issue Resolution via Comprehensive Repository Explorationcited by 58
  25. 2024Lingma SWE-GPT: An Open Development-Process-Centric Language Model for Automated Software Improvementcited by 57
  26. 2025OpenCodeInstruct: A Large-scale Instruction Tuning Dataset for Code LLMscited by 57
  27. 2024Compression Represents Intelligence Linearlycited by 56
  28. 2024OriGen:Enhancing RTL Code Generation with Code-to-Code Augmentation and Self-Reflectioncited by 56
  29. 2024JetMoE: Reaching Llama2 Performance with 0.1M Dollarscited by 55
  30. 2025CodeI/O: Condensing Reasoning Patterns via Code Input-Output Predictioncited by 54
  31. 2025DataSciBench: An LLM Agent Benchmark for Data Sciencecited by 53
  32. 2024What's Wrong with Your Code Generated by Large Language Models? An Extensive Studycited by 52
  33. 2024Long Code Arena: a Set of Benchmarks for Long-Context Code Modelscited by 48
  34. 2025Code to Think, Think to Code: A Survey on Code-Enhanced Reasoning and Reasoning-Driven Code Intelligence in LLMscited by 47
  35. 2025MegaMath: Pushing the Limits of Open Math Corporacited by 47
  36. 2025Seed-Coder: Let the Code Model Curate Data for Itselfcited by 46
  37. 2024Fishing for Magikarp: Automatically Detecting Under-trained Tokens in Large Language Modelscited by 45
  38. 2024AgileCoder: Dynamic Collaborative Agents for Software Development based on Agile Methodologycited by 45

50 most cited of 484 All 484 (JSON)