mlsanityData to 14 Oct 2026

TreeDiff: AST-Guided Code Generation with Diffusion LLMs

arXiv 2508.01473 ↗First posted 2 Aug 2025cs.CLcited by 11

Not on the map. Why?

Abstract (arXiv)

Code generation is increasingly critical for real-world applications. Still, diffusion-based large language models continue to struggle with this demand. Unlike free-form text, code requires syntactic precision; even minor structural inconsistencies can render a program non-executable. Existing diffusion-based large language models rely on random token masking for corruption, leading to two key failures: they lack awareness of syntactic boundaries during the iterative denoising process, and they fail to capture the long-range hierarchical dependencies essential for program correctness. We propose TreeDiff to address both issues. Specifically, we propose a syntax-aware diffusion framework that incorporates structural priors from Abstract Syntax Tree (AST) into the corruption process. Instead of masking individual tokens at random, we selectively mask tokens belonging to key AST nodes. By aligning the corruption process with the underlying structure of code, our method encourages the model to internalize the compositional nature of programming languages, enabling it to reconstruct programs that respect grammatical boundaries and capture long-range dependencies. Our method achieves a 13.3% relative improvement over the random masking training method, demonstrating its effectiveness in code generation task by leveraging underlying structures.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Builds on 31 of its 49 references that are in the map. First posted on arXiv 2 Aug 2025.

Builds on

31 of 49 references.

References · 31

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  3. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  5. 2023Code Llama: Open Foundation Models for Codecited by 2,139
  6. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,719
  7. 2022Competition-Level Code Generation with AlphaCodecited by 1,462
  8. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,391
  9. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,027
  10. 2024Simple and Effective Masked Diffusion Language Modelscited by 848
  11. 2024A Survey on Large Language Models for Code Generationcited by 658
  12. 2025Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Modelscited by 409
19 more
  1. 2025Demystifying Long Chain-of-Thought Reasoning in LLMscited by 326
  2. 2025DiffuCoder: Understanding and Improving Masked Diffusion Models for Code Generationcited by 187
  3. 2018code2seq: Generating Sequences from Structured Representations of Codecited by 179
  4. 2025OpenCodeReasoning: Advancing Data Distillation for Competitive Codingcited by 113
  5. 2017Abstract Syntax Networks for Code Generation and Semantic Parsingcited by 92
  6. 2023RewriteLM: An Instruction-Tuned Large Language Model for Text Rewritingcited by 67
  7. 2025Recent Advances in Large Langauge Model Benchmarks against Data Contamination: From Static to Dynamic Evaluationcited by 38
  8. 2024AST-T5: Structure-Aware Pretraining for Code Generation and Understandingcited by 33
  9. 2023StudentEval: A Benchmark of Student-Written Prompts for Large Language Models of Codecited by 32
  10. 2025Uncertainty-Guided Chain-of-Thought for Code Generation with LLMscited by 32
  11. 2023CodeFusion: A Pre-trained Diffusion Model for Code Generationcited by 31
  12. 2024CodeJudge-Eval: Can Large Language Models be Good Judges in Code Understanding?cited by 30
  13. 2024Reassessing Layer Pruning in LLMs: New Insights and Methodscited by 24
  14. 2025Dynamic Benchmarking of Reasoning Capabilities in Code Large Language Models Under Data Contaminationcited by 23
  15. 2025Bridging the Editing Gap in LLMs: FineEdit for Precise and Targeted Text Modificationscited by 21
  16. 2025Chain-of-Thought Matters: Improving Long-Context Language Models with Reasoning Path Supervisioncited by 21
  17. 2024PPM: Automated Generation of Diverse Programming Problems for Benchmarking Code Generation Modelscited by 8
  18. 2025From LLM-anation to LLM-orchestrator: Coordinating Small Models for Data Labelingcited by 8
  19. 2025Facilitating Long Context Understanding via Supervised Chain-of-Thought Reasoningcited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 7.

11 papers.

  1. 2025LaDiR: Latent Diffusion Enhances LLMs for Text Reasoningcited by 29
  2. 2025Exploring the Potential of Diffusion Large Language Models in Code Generationcited by 16
  3. 2026Lookahead-then-Verify: Reliable Constrained Decoding for Diffusion LLMs under Context-Free Grammarscited by 7
  4. 2026LaDi-RL: Latent Diffusion Reasoning Prevents Entropy Collapse in Reinforcement Learningcited by 7
  5. 2025WavefrontDiffusion: Dynamic Decoding Schedule for Improved Reasoningcited by 4
  6. 2026From AR to Diffusion: Efficiently Adapting Large Language Models with Strictly Causal and Elastic Horizonscited by 3
  7. 2025DiffuTester: Accelerating Unit Test Generation for Diffusion LLMs via Mining Structural Patterncited by 2
  8. 2026AnCoder: Anchored Code Generation via Discrete Diffusion Modelscited by 1
  9. 2026LLMs + Security = Troublecited by 0
  10. 2026Beyond Execution: Static-Analysis Rewards and Hint-Conditioned Diffusion RL for Code Generationcited by 0
  11. 2026Beyond the Autoregressive Horizon: A Comprehensive Survey of Diffusion Models, World Modelling, and State Space Models for Codecited by 0