mlsanityData to 7 Oct 2026

Agents of Diffusion: Enhancing Diffusion Language Models with Multi-Agent Reinforcement Learning for Structured Data Generation (Extended Version)

arXiv 2601.07152 ↗First posted 12 Jan 2026cs.MAcited by 0

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

Generating high-quality structured data such as JSON records, remains a fundamental challenge for large language models (LLMs), particularly when semantic richness must coexist with strict schema adherence. While autoregressive LLMs offer strong structural consistency, they often struggle with semantic variation and output diversity. In contrast, diffusion language models (DLMs) introduce powerful mechanisms for semantic richness and bidirectional decoding, yet lack the inductive biases needed for reliable structure preservation. We present Agents of Diffusion (AoD), a novel framework that unifies the generative flexibility of DLMs with the reasoning capabilities of autoregressive models through language-mediated reinforcement learning. AoD frames structured text generation as a multi-agent alignment process, where a prompt optimization agent collaborates with a judge agent to iteratively guide a DLM using natural language feedback. This approach enables controllable, schema-consistent generation without modifying model parameters or relying on handcrafted constraints. AoD advances the state of controllable generation by demonstrating that diffusion models, when supervised by cooperative agents, can achieve both high semantic novelty and structural fidelity. Across multiple structured data benchmarks, AoD consistently outperforms diffusion and autoregressive baselines, establishing a new path forward for structure-aware, diversity-enhanced text synthesis.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 32 of its 48 references that are in the map. First posted on arXiv 12 Jan 2026.

Builds on

32 of 48 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025Multi-Agent Collaboration Mechanisms: A Survey of LLMs10 Jan

Other references · 31

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  6. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  7. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  8. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  9. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,906
  10. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,739
  11. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,006
  12. 2022Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Taskscited by 858
19 more
  1. 2019CTRL: A Conditional Transformer Language Model for Controllable Generationcited by 754
  2. 2022Large Language Models Can Self-Improvecited by 656
  3. 2023Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolutioncited by 521
  4. 2022DiffuSeq: Sequence to Sequence Text Generation with Diffusion Modelscited by 439
  5. 2022Understanding Diffusion Models: A Unified Perspectivecited by 337
  6. 2023Large Language Model as Attributed Training Data Generator: A Tale of Diversity and Biascited by 230
  7. 2022ZeroGen: Efficient Zero-shot Learning via Dataset Generationcited by 209
  8. 2021Autoregressive Diffusion Modelscited by 208
  9. 2023Mixed-Type Tabular Data Synthesis with Score-based Diffusion in Latent Spacecited by 183
  10. 2023Large Language Models as Evolutionary Optimizerscited by 122
  11. 2024Survey on Large Language Model-Enhanced Reinforcement Learning: Concept, Taxonomy, and Methodscited by 114
  12. 2024Guiding LLMs The Right Way: Fast, Non-Invasive Constrained Generationcited by 99
  13. 2023Exploiting Asymmetry for Synthetic Training Data Generation: SynthIE and the Case of Information Extractioncited by 89
  14. 2024Comprehensive Exploration of Synthetic Data Generation: A Surveycited by 59
  15. 2023Controlled Text Generation via Language Model Arithmeticcited by 51
  16. 2024CodecLM: Aligning Language Models with Tailored Synthetic Datacited by 39
  17. 2023Generating Faithful Synthetic Data with Large Language Models: A Case Study in Computational Social Sciencecited by 37
  18. 2024DiffLM: Controllable Synthetic Data Generation via Diffusion Language Modelscited by 9
  19. 2025A Generalized Theory of Mixup for Structure-Preserving Synthetic Datacited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Unveiling the Potential of Diffusion Large Language Model in Controllable Generation
  2. 2026Diffusion Language Models: An Experimental Analysis
  3. 2026Breaking the Factorization Barrier in Diffusion Language Models
  4. 2024Scaling Diffusion Language Models via Adaptation from Autoregressive Models
  5. 2026From Position Risks to Block Survival: Faster Generation for Diffusion Language Models
  6. 2025Breaking AR's Sampling Bottleneck: Provable Acceleration via Diffusion Language Models
  7. 2026Top 10 Open Challenges Steering the Future of Diffusion Language Model and Its Variants
  8. 2026Hierarchical Continuous Diffusion Language Models