mlsanityData to 7 Oct 2026

CodeFusion: A Pre-trained Diffusion Model for Code Generation

arXiv 2310.17680 ↗First posted 26 Oct 2023cs.SE cs.AI cs.CLcited by 31

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

Imagine a developer who can only change their last line of code, how often would they have to start writing a function from scratch before it is correct? Auto-regressive models for code generation from natural language have a similar limitation: they do not easily allow reconsidering earlier tokens generated. We introduce CodeFusion, a pre-trained diffusion code generation model that addresses this limitation by iteratively denoising a complete program conditioned on the encoded natural language. We evaluate CodeFusion on the task of natural language to code generation for Bash, Python, and Microsoft Excel conditional formatting (CF) rules. Experiments show that CodeFusion (75M parameters) performs on par with state-of-the-art auto-regressive systems (350M-175B parameters) in top-1 accuracy and outperforms them in top-3 and top-5 accuracy due to its better balance in diversity versus quality.

Key facts. Cited by 31 papers in the map (data through 7 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 27 of its 36 references that are in the map. First posted on arXiv 26 Oct 2023.

Builds on

27 of 36 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023AR-Diffusion: Auto-Regressive Diffusion Model for Text Generation16 May
  2. 2022Text Generation with Diffusion Language Models: A Pre-training Approach with Continuous Paragraph Denoise22 Dec

Other references · 25

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  6. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  7. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  8. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  9. 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,286
  10. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,006
  11. 2021CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generationcited by 890
  12. 2022CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesiscited by 876
13 more
  1. 2023WizardCoder: Empowering Code Large Language Models with Evol-Instructcited by 728
  2. 2022InCoder: A Generative Model for Code Infilling and Synthesiscited by 492
  3. 2021PICARD: Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Modelscited by 402
  4. 2022A Systematic Evaluation of Large Language Models of Codecited by 360
  5. 2023CodeT5+: Open Code Large Language Models for Code Understanding and Generationcited by 327
  6. 2016Diverse Beam Search: Decoding Diverse Solutions from Neural Sequence Modelscited by 299
  7. 2018Learning to Mine Aligned Code and Natural Language Pairs from Stack Overflowcited by 165
  8. 2023CodeBERTScore: Evaluating Code Generation with Pretrained Models of Codecited by 101
  9. 2018NL2Bash: A Corpus and Semantic Parser for Natural Language Interface to the Linux Operating Systemcited by 97
  10. 2021Non-Autoregressive Text Generation with Pre-trained Language Modelscited by 36
  11. 2023An Empirical Comparison of Pre-Trained Models of Source Codecited by 25
  12. 2021Toward Code Generation: A Survey and Lessons from Semantic Parsingcited by 8
  13. 2023TransCoder: Towards Unified Transferable Code Representation Learning Inspired by Human Skillscited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 8; 2025: 15; 2026: 7.

31 papers.

Sort:

By area:

  1. 2024A Survey on Large Language Models for Code Generationcited by 646
  2. 2025DiffuCoder: Understanding and Improving Masked Diffusion Models for Code Generationcited by 184
  3. 2023LaMPilot: An Open Benchmark Dataset for Autonomous Driving with Language Model Programscited by 45
  4. 2024Logits of API-Protected LLMs Leak Proprietary Informationcited by 38
  5. 2025LaDiR: Latent Diffusion Enhances LLMs for Text Reasoningcited by 25
  6. 2025From Bits to Rounds: Parallel Decoding with Exploration for Diffusion Language Modelscited by 22
  7. 2025Proving the Coding Interview: A Benchmark for Formally Verified Code Generationcited by 18
  8. 2025Exploring the Potential of Diffusion Large Language Models in Code Generationcited by 16
  9. 2026DreamOn: Diffusion Language Models For Code Infilling Beyond Fixed-size Canvascited by 16
  10. 2025TreeDiff: AST-Guided Code Generation with Diffusion LLMscited by 11
  11. 2024Pragmatic Competence Evaluation of Large Language Models for the Korean Languagecited by 8
  12. 2024FALCON: Feedback-driven Adaptive Long/short-term memory reinforced Coding Optimization systemcited by 8
19 more
  1. 2025Large Language Models are Qualified Benchmark Builders: Rebuilding Pre-Training Datasets for Advancing Code Intelligence Taskscited by 6
  2. 2025MOCHA: Are Code Language Models Robust Against Multi-Turn Malicious Coding Prompts?cited by 5
  3. 2024A Survey on Large Language Models from Concept to Implementationcited by 4
  4. 2025Text-Diffusion Red-Teaming of Large Language Models: Unveiling Harmful Behaviors with Proximity Constraintscited by 4
  5. 2025A Neurosymbolic Agent System for Compositional Visual Reasoningcited by 4
  6. 2025A Novel Diffusion Model for Pairwise Geoscience Data Generation with Unbalanced Training Datasetcited by 3
  7. 2025Diffusion is a code repair operator and generatorcited by 3
  8. 2026Why Gaussian Diffusion Models Fail on Discrete Data and How to Prevent It?cited by 3
  9. 2025System Log Parsing with Large Language Models: A Reviewcited by 2
  10. 2026Distilled Continuous Diffusion Language Models Can Write Code in Few Steps---or Onecited by 2
  11. 2025MoE-DiffuSeq: Enhancing Long-Document Diffusion Models with Sparse Attention and Mixture of Expertscited by 1
  12. 2026Autoregressive, Yet Revisable: In Decoding Revision for Secure Code Generationcited by 1
  13. 2026DICE: Diffusion Large Language Models Excel at Generating CUDA Kernelscited by 1
  14. 2024A Survey on the Real Power of ChatGPTcited by 0
  15. 2024Thesis proposal: Are We Losing Textual Diversity to Natural Language Processing?cited by 0
  16. 2024Eliciting Instruction-tuned Code Language Models' Capabilities to Utilize Auxiliary Function for Code Generationcited by 0
  17. 2025Tabularis Formatus: Predictive Formatting for Tablescited by 0
  18. 2026Beyond the Autoregressive Horizon: A Comprehensive Survey of Diffusion Models, World Modelling, and State Space Models for Codecited by 0
  19. 2026Fine-Tuning Qwen3-27B for C-to-Rust Code Translation: A Three-Stage Curriculum of Pretraining, Debugging-Aware SFT, and Task-Specific SFTcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Constrained Code Generation with Discrete Diffusion
  2. 2023Improving Code Generation by Training with Natural Language Feedback
  3. 2026AnCoder: Anchored Code Generation via Discrete Diffusion Models
  4. 2025Dream-Coder 7B: An Open Diffusion Language Model for Code
  5. 2026ShortCoder: Knowledge-Augmented Syntax Optimization for Token-Efficient Code Generation
  6. 2024CodeLutra: Boosting LLM Code Generation via Preference-Guided Refinement
  7. 2024Comments as Natural Logic Pivots: Improve Code Generation via Comment Perspective
  8. 2026CodeAlchemy: Synthetic Code Rewriting at Scale