mlsanityData to 7 Oct 2026

CoDA: Coding LM via Diffusion Adaptation

arXiv 2510.03270 ↗First posted 27 Sep 2025cs.LG cs.AIcited by 4

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

Diffusion language models promise bidirectional context and infilling capabilities that autoregressive coders lack, yet practical systems remain heavyweight. We introduce CoDA, a 1.7B-parameter diffusion coder trained on TPU with a fully open-source training pipeline. CoDA pairs large-scale diffusion pre-training with code-centric mid-training and instruction tuning, enabling confidence-guided sampling that keeps inference latency competitive. On Humaneval, MBPP, and EvalPlus, CoDA-1.7B-Instruct matches or surpasses diffusion models up to 7B parameters. Our release includes model checkpoints, evaluation harnesses, and TPU training pipelines to accelerate research on lightweight diffusion-based coding assistants.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 24 of its 29 references that are in the map. First posted on arXiv 27 Sep 2025.

Builds on

24 of 29 references.

Closest ancestors · 9

Same direction, newest first

  1. 2025Revolutionizing Reinforcement Learning Framework for Diffusion Large Language Models8 Sep
  2. 2025Dream 7B: Diffusion Large Language Models21 Aug
  3. 2025Seed Diffusion: A Large-Scale Diffusion Language Model with High-Speed Inference4 Aug
  4. 2025DiffuCoder: Understanding and Improving Masked Diffusion Models for Code Generation25 Jun
  5. 2025Large Language Diffusion Models14 Feb
  6. 2024Scaling up Masked Diffusion Models on Text24 Oct
  7. 2024Scaling Diffusion Language Models via Adaptation from Autoregressive Models23 Oct
  8. 2024Simplified and Generalized Masked Diffusion for Discrete Data6 Jun
  9. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution25 Oct

Other references · 15

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2020Denoising Diffusion Implicit Modelscited by 9,655
  4. 2025Qwen3 Technical Reportcited by 8,195
  5. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  6. 2021Program Synthesis with Large Language Modelscited by 3,509
  7. 2024Qwen2.5-Coder Technical Reportcited by 1,151
  8. 2018Adafactor: Adaptive Learning Rates with Sublinear Memory Costcited by 896
  9. 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decodingcited by 404
  10. 2024DataComp-LM: In search of the next generation of training sets for language modelscited by 381
  11. 2019Analysing Mathematical Reasoning Abilities of Neural Modelscited by 331
  12. 2025Mercury: Ultra-Fast Language Models Based on Diffusioncited by 225
3 more
  1. 2024RedPajama: an Open Dataset for Training Large Language Modelscited by 178
  2. 2025d1: Scaling Reasoning in Diffusion Large Language Models via Reinforcement Learningcited by 178
  3. 2023OpenWebMath: An Open Dataset of High-Quality Mathematical Web Textcited by 151

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026AnCoder: Anchored Code Generation via Discrete Diffusion Modelscited by 1
  2. 2026Prompt Optimization Via Diffusion Language Modelscited by 1
  3. 2026On the Quantization Robustness of Diffusion Language Models in Coding Benchmarkscited by 0
  4. 2026dQwen3.5: Hybrid-Attention Diffusion Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025From Next-Token to Next-Block: A Principled Adaptation Path for Diffusion LLMs
  2. 2025TESS 2: A Large-Scale Generalist Diffusion Language Model
  3. 2026CoDAR: Continuous Diffusion Language Models are More Powerful Than You Think
  4. 2026CoDR: Training-Free Confidence-Drift Remasking for Diffusion Language Models
  5. 2025Dream-Coder 7B: An Open Diffusion Language Model for Code
  6. 2026Adaptive Block Diffusion: Resolving Training-Inference Mismatch in Diffusion Language Models
  7. 2026CAI-DLLM: Convergence Aware Inference for Diffusion Language Models
  8. 2026DreamOn: Diffusion Language Models For Code Infilling Beyond Fixed-size Canvas