mlsanityData to 13 Oct 2026

C2DLM: Causal Concept-Guided Diffusion Large Language Models

arXiv 2511.22146 ↗First posted 27 Nov 2025cs.CLcited by 1

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

Autoregressive (AR) language models and Diffusion Language Models (DLMs) constitute the two principal paradigms of large language models. However, both paradigms suffer from insufficient reasoning capabilities. Human reasoning inherently relies on causal knowledge and thought, which are reflected in natural language. But in the AR paradigm, language is modeled as next token prediction (a strictly left-to-right, token-by-token order), whereas natural language itself exhibits more flexible causal structures. In the DLM paradigm, the attention mechanism is fully connected, which entirely disregards causal order. To fill this gap, we propose a underlineCausal underlineConcept-Guided underlineDiffusion underlineLanguage underlineModel (C2DLM). Starting from DLM's fully connected attention, C2DLM first obtains a concept-level causal graph from the teacher model, and then explicitly guides attention to learn causal relationships between concepts. By focusing on causal relationships and avoiding interference from difficult subgoals involving causal inversion, C2DLM improves 12% with about 3.2 times training speedup in the COT-OrderPerturb task, and achieves an average gain of 1.31% across six downstream reasoning tasks. More details in the repository ~hrefhttps://github.com/Kairong-Han/C-2-DLMhere.

Key facts. Cited by 1 paper in the map (data through 13 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 33 of its 54 references that are in the map. First posted on arXiv 27 Nov 2025.

Builds on

33 of 54 references.

Closest ancestors · 11

Same direction, newest first

  1. 2025Inpainting-Guided Policy Optimization for Diffusion Large Language Models12 Sep
  2. 2025Dream 7B: Diffusion Large Language Models21 Aug
  3. 2025A Survey on Diffusion Language Models14 Aug
  4. 2025Time Is a Feature: Exploiting Temporal Dynamics in Diffusion Language Models12 Aug
  5. 2025Diffusion LLMs Can Do Faster-Than-AR Inference via Discrete Diffusion Forcing8 Aug
  6. 2025wd1: Weighted Policy Optimization for Reasoning in Diffusion Language Models7 Jul
  7. 2025FlashDLM: Accelerating Diffusion Language Model Inference via Efficient KV Caching and Guided Diffusion27 May
  8. 2025Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models12 Mar
  9. 2025Large Language Diffusion Models14 Feb
  10. 2025Train for the Worst, Plan for the Best: Understanding Token Ordering in Masked Diffusions10 Feb
  11. 2024Beyond Autoregression: Discrete Diffusion for Complex Reasoning and Planning18 Oct

Other references · 22

  1. 2021Training Verifiers to Solve Math Word Problemscited by 9,916
  2. 2025Qwen3 Technical Reportcited by 8,510
  3. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,798
  4. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,586
  5. 2023A Survey of Large Language Modelscited by 2,559
  6. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,137
  7. 2023Gemini: A Family of Highly Capable Multimodal Modelscited by 848
  8. 2024DeepSeek-V3 Technical Reportcited by 777
  9. 2022A Survey on In-context Learningcited by 719
  10. 2023Measuring Faithfulness in Chain-of-Thought Reasoningcited by 689
  11. 2025GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Modelscited by 456
  12. 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decodingcited by 409
10 more
  1. 2024Massive Activations in Large Language Modelscited by 283
  2. 2025LLaDA 1.5: Variance-Reduced Preference Optimization for Large Language Diffusion Modelscited by 275
  3. 2024When Attention Sink Emerges in Language Models: An Empirical Viewcited by 202
  4. 2025Survey on Evaluation of LLM-based Agentscited by 200
  5. 2025d1: Scaling Reasoning in Diffusion Large Language Models via Reinforcement Learningcited by 182
  6. 2024LLMs Can't Plan, But Can Help Planning in LLM-Modulo Frameworkscited by 179
  7. 2023Causal Parrots: Large Language Models May Talk Causality But Are Not Causalcited by 150
  8. 2025BranchGRPO: Stable and Efficient GRPO with Structured Branching in Diffusion Modelscited by 61
  9. 2025Accelerating Diffusion Large Language Models with SlowFast Sampling: The Three Golden Principlescited by 43
  10. 2024Causality for Large Language Modelscited by 24

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Towards Efficient Reasoning: Learning Causal Shortcuts for Diffusion Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Diffusion Language Models: An Experimental Analysis
  2. 2026Reasoning with Latent Tokens in Diffusion Language Models
  3. 2025Unveiling Causal Reasoning in Large Language Models: Reality or Mirage?
  4. 2025Beyond Surface Reasoning: Unveiling the True Long Chain-of-Thought Capacity of Diffusion Large Language Models
  5. 2026Hierarchical Continuous Diffusion Language Models
  6. 2026Data-Efficient Autoregressive-to-Diffusion Language Models via On-Policy Distillation
  7. 2025Efficient-DLM: From Autoregressive to Diffusion Language Models, and Beyond in Speed
  8. 2024C2P: Featuring Large Language Models with Causal Reasoning