mlsanityData to 7 Oct 2026

Seed-Coder: Let the Code Model Curate Data for Itself

arXiv 2506.03524 ↗First posted 4 Jun 2025cs.CL cs.SEcited by 46

Not on the map. Why?

Abstract (arXiv)

Code data in large language model (LLM) pretraining is recognized crucial not only for code-related tasks but also for enhancing general intelligence of LLMs. Current open-source LLMs often heavily rely on human effort to produce their code pretraining data, such as employing hand-crafted filtering rules tailored to individual programming languages, or using human-annotated data to train quality filters. However, these approaches are inherently limited in scalability, prone to subjective biases, and costly to extend and maintain across diverse programming languages. To address these challenges, we introduce Seed-Coder, a series of open-source LLMs comprising base, instruct and reasoning models of 8B size, minimizing human involvement in data construction. Our code pretraining data is produced by a model-centric data pipeline, which predominantly leverages LLMs for scoring and filtering code data. The instruct model is further trained via supervised fine-tuning and preference optimization, and the reasoning model leverages Long-Chain-of-Thought (LongCoT) reinforcement learning to improve multi-step code reasoning. Seed-Coder achieves state-of-the-art results among open-source models of similar size and even surpasses some much larger models, demonstrating superior performance in code generation, code completion, code editing, code reasoning, and software engineering tasks.

Key facts. Cited by 46 papers in the map (data through 7 Oct 2026). Builds on 34 of its 57 references that are in the map. First posted on arXiv 4 Jun 2025.

Builds on

34 of 57 references.

References · 34

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  4. 2021Program Synthesis with Large Language Modelscited by 3,509
  5. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
  6. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 1,980
  7. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 1,906
  8. 2022Competition-Level Code Generation with AlphaCodecited by 1,427
  9. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,332
  10. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,154
  11. 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,093
  12. 2024OpenHands: An Open Platform for AI Software Developers as Generalist Agentscited by 956
22 more
  1. 2025Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?cited by 948
  2. 2016Bag of Tricks for Efficient Text Classificationcited by 869
  3. 2023StarCoder: may the source be with you!cited by 790
  4. 2021Deduplicating Training Data Makes Language Models Bettercited by 705
  5. 2024BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructionscited by 527
  6. 2022InCoder: A Generative Model for Code Infilling and Synthesiscited by 492
  7. 2024StarCoder 2 and The Stack v2: The Next Generationcited by 484
  8. 2023RepoCoder: Repository-Level Code Completion Through Iterative Retrieval and Generationcited by 357
  9. 2024DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligencecited by 325
  10. 2024CRUXEval: A Benchmark for Code Reasoning, Understanding and Executioncited by 294
  11. 2022Efficient Training of Language Models to Fill in the Middlecited by 248
  12. 2023CrossCodeEval: A Diverse and Multilingual Benchmark for Cross-File Code Completioncited by 181
  13. 2023SantaCoder: don't reach for the stars!cited by 169
  14. 2022Multi-lingual Evaluation of Code Generation Modelscited by 158
  15. 2025CodeElo: Benchmarking Competition-level Code Generation of LLMs with Human-comparable Elo Ratingscited by 109
  16. 2023#InsTag: Instruction Tagging for Analyzing Supervised Fine-tuning of Large Language Modelscited by 108
  17. 2024Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilizationcited by 97
  18. 2024Unveiling and Harnessing Hidden Attention Sinks: Enhancing Large Language Models without Training through Attention Calibrationcited by 70
  19. 2024FullStack Bench: Evaluating LLMs as Full Stack Coderscited by 56
  20. 2023Can It Edit? Evaluating the Ability of Large Language Models to Follow Code Editing Instructionscited by 46
  21. 2024MHPP: Exploring the Capabilities and Limitations of Language Models Beyond Basic Code Generationcited by 25
  22. 2024NaturalCodeBench: Examining Coding Performance Mismatch on HumanEval and Natural User Promptscited by 19

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 16; 2026: 30.

46 papers.

Sort:

By area:

  1. 2025Seed Diffusion: A Large-Scale Diffusion Language Model with High-Speed Inferencecited by 173
  2. 2025NL2Repo-Bench: Towards Long-Horizon Repository Generation Evaluation of Coding Agentscited by 56
  3. 2026SWE-Pruner: Self-Adaptive Context Pruning for Coding Agentscited by 40
  4. 2025AutoTriton: Automatic Triton Programming with Reinforcement Learning in LLMscited by 38
  5. 2025LongCodeZip: Compress Long Context for Code Language Modelscited by 38
  6. 2025ArtifactsBench: Bridging the Visual-Interactive Gap in LLM Code Generation Evaluationcited by 35
  7. 2026LongCLI-Bench: A Preliminary Benchmark and Study for Long-horizon Agentic Programming in Command-Line Interfacescited by 29
  8. 2025Let It Flow: Agentic Crafting on Rock and Roll, Building the ROME Model within an Open Agentic Learning Ecosystemcited by 28
  9. 2025TritonRL: Training LLMs to Think and Code Triton Without Cheatingcited by 24
  10. 2026BeyondSWE: Can Current Code Agent Survive Beyond Single-Repo Bug Fixing?cited by 20
  11. 2025Exploring the Potential of Diffusion Large Language Models in Code Generationcited by 16
  12. 2026DreamOn: Diffusion Language Models For Code Infilling Beyond Fixed-size Canvascited by 16
34 more
  1. 2025From Code Foundation Models to Agents and Applications: A Comprehensive Survey and Practical Guide to Code Intelligencecited by 13
  2. 2026Stable-DiffCoder: Pushing the Frontier of Code Diffusion Large Language Modelcited by 12
  3. 2025Are We Ready for RL in Text-to-3D Generation? A Progressive Investigationcited by 8
  4. 2025CodeBoost: Boosting Code LLMs by Squeezing Knowledge from Code Snippets with RLcited by 7
  5. 2026Reward Modeling from Natural Language Human Feedbackcited by 7
  6. 2026LaDi-RL: Latent Diffusion Reasoning Prevents Entropy Collapse in Reinforcement Learningcited by 7
  7. 2025FeatBench: Towards More Realistic Evaluation of Feature-level Code Generationcited by 6
  8. 2025Agnostics: Learning to Code in Any Programming Language via Reinforcement with a Universal Learning Environmentcited by 5
  9. 2025Your thoughts tell who you are: Characterize the reasoning patterns of LRMscited by 4
  10. 2026Your Code Agent Can Grow Alongside You with Structured Memorycited by 4
  11. 2026InCoder-32B-Thinking: Industrial Code World Model for Thinkingcited by 4
  12. 2026Latent Reasoning with Normalizing Flowscited by 4
  13. 2026AdaExplore: Failure-Driven Adaptation and Diversity-Preserving Search for Efficient Kernel Generationcited by 3
  14. 2026AscendOptimizer: Episodic Agent for Ascend NPU Operator Optimizationcited by 2
  15. 2025The Cream Rises to the Top: Efficient Reranking Method for Verilog Code Generationcited by 1
  16. 2026Monotonic Reference-Free Refinement for Autoformalizationcited by 1
  17. 2026Rethinking Scientific Modeling: Toward Physically Consistent and Simulation-Executable Programmatic Generationcited by 1
  18. 2026DICE: Diffusion Large Language Models Excel at Generating CUDA Kernelscited by 1
  19. 2026Steerable Instruction Following Coding Data Synthesis with Actor-Parametric Schema Co-Evolutioncited by 1
  20. 2026Bridging Generation and Training: A Systematic Review of Quality Issues in LLMs for Codecited by 1
  21. 2026From Tokens to Regions: CUDA-Sensitive Instruction Tuning for GPU Kernel Generationcited by 1
  22. 2026CODEBLOCK: Learning to Supervise Code at the Right Granularitycited by 1
  23. 2026Efficient Test-Time Optimization for Multi-Agent Proof Autoformalizationcited by 1
  24. 2025SCoGen: Scenario-Centric Graph-Based Synthesis of Real-World Code Problemscited by 0
  25. 2026Beyond Text-to-SQL: Can LLMs Really Debug Enterprise ETL SQL?cited by 0
  26. 2026Synthesizing File-Level Data for Unit Test Generation with Chain-of-Thoughts via Self-Debuggingcited by 0
  27. 2026IESR:Efficient MCTS-Based Modular Reasoning for Text-to-SQL with Large Language Modelscited by 0
  28. 2026Detecting RLVR Training Data via Structural Convergence of Reasoningcited by 0
  29. 2026Mellum2 Technical Reportcited by 0
  30. 2026Diagnosing Knowledge Gaps in LLM Tool Use: An Agentic Benchmark for Novel API Acquisitioncited by 0
  31. 2026LLM Evolution as an Industry-Scale Ecosystem: A Lifecycle Perspective on Continual Learningcited by 0
  32. 2026RTLCurator: Label-Efficient Data Curation for RTL Generationcited by 0
  33. 2026CodeTS: Verifiable Text-to-Time Series Generation via Executable Codecited by 0
  34. 2026CodeSkill: Latent Skill Abstraction for Long-Horizon Code Agentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Increasing LLM Coding Capabilities through Diverse Synthetic Coding Tasks
  2. 2025Every Sample Matters: Leveraging Mixture-of-Experts and High-Quality Data for Efficient and Accurate Code LLM
  3. 2024Improving the Ability of Pre-trained Language Model by Imparting Large Language Model's Experience ↗
  4. 2025Rewriting Pre-Training Data Boosts LLM Performance in Math and Code
  5. 2024InverseCoder: Self-improving Instruction-Tuned Code LLMs with Inverse-Instruct
  6. 2024CodeLutra: Boosting LLM Code Generation via Preference-Guided Refinement
  7. 2025HardTests: Synthesizing High-Quality Test Cases for LLM Coding
  8. 2025Synthetic Data Generation Using Large Language Models: Advances in Text and Code