mlsanityData to 7 Oct 2026

Discrete Tilt Matching

arXiv 2604.18739 ↗First posted 20 Apr 2026cs.LG stat.MLcited by 0

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

Masked diffusion large language models (dLLMs) are a promising alternative to autoregressive generation. While reinforcement learning (RL) methods have recently been adapted to dLLM fine-tuning, their objectives typically depend on sequence-level marginal likelihoods, which are intractable for masked diffusion models. To address this, we derive Discrete Tilt Matching (DTM), a likelihood-free method that recasts dLLM fine-tuning as state-level matching of local unmasking posteriors under reward tilting. DTM takes the form of a weighted cross-entropy objective with explicit minimizer, and admits control variates that improve training stability. On a synthetic maze-planning task, we analyze how DTM's annealing schedule and control variates affect training stability and prevent mode collapse. At scale, fine-tuning LLaDA-8B-Instruct with DTM yields strong gains on Sudoku and Countdown while remaining competitive on MATH500 and GSM8K.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 27 of its 34 references that are in the map. First posted on arXiv 20 Apr 2026.

Builds on

27 of 34 references.

Closest ancestors · 13

Same direction, newest first

  1. 2025SPG: Sandwiched Policy Gradient for Masked Diffusion Language Models10 Oct
  2. 2025Improving Reasoning for Diffusion Language Models via Group Diffusion Policy Optimization9 Oct
  3. 2025d2: Improving Reasoning in Diffusion Language Models via Trajectory Likelihood Estimation25 Sep
  4. 2025Dream-Coder 7B: An Open Diffusion Language Model for Code1 Sep
  5. 2025Any-Order Flexible Length Masked Diffusion31 Aug
  6. 2025wd1: Weighted Policy Optimization for Reasoning in Diffusion Language Models7 Jul
  7. 2025DiffuCoder: Understanding and Improving Masked Diffusion Models for Code Generation25 Jun
  8. 2025Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models12 Mar
  9. 2025Large Language Diffusion Models14 Feb
  10. 2025Train for the Worst, Plan for the Best: Understanding Token Ordering in Masked Diffusions10 Feb
  11. 2024Simple and Effective Masked Diffusion Language Models11 Jun
  12. 2024Simplified and Generalized Masked Diffusion for Discrete Data6 Jun
1 more
  1. 2022SSD-LM: Semi-autoregressive Simplex-based Diffusion Language Model for Text Generation and Modular Control31 Oct

Other references · 14

  1. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  4. 2023Let's Verify Step by Stepcited by 3,598
  5. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  6. 2022MaskGIT: Masked Generative Image Transformercited by 1,082
  7. 2024Generative Flows on Discrete State-Spaces: Enabling Multimodal Flows with Applications to Protein Co-Designcited by 294
  8. 2025LLaDA 1.5: Variance-Reduced Preference Optimization for Large Language Diffusion Modelscited by 273
  9. 2025MMaDA: Multimodal Large Diffusion Language Modelscited by 241
  10. 2025d1: Scaling Reasoning in Diffusion Large Language Models via Reinforcement Learningcited by 178
  11. 2024Flow Matching with General Discrete Paths: A Kinetic-Optimal Perspectivecited by 54
  12. 2025Tilt Matching for Scalable Sampling and Fine-Tuningcited by 17
2 more
  1. 2025Reinforcing Diffusion Models by Direct Group Preference Optimizationcited by 13
  2. 2026Stable-DiffCoder: Pushing the Frontier of Code Diffusion Large Language Modelcited by 12

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Enhancing Reasoning for Diffusion LLMs via Distribution Matching Policy Optimization
  2. 2025Advantage Weighted Matching: Aligning RL with Pretraining in Diffusion Models
  3. 2026Scaling Reinforcement Learning for Diffusion Models via Velocity Matching
  4. 2025Inpainting-Guided Policy Optimization for Diffusion Large Language Models
  5. 2026DARE: Diffusion Large Language Models Alignment and Reinforcement Executor
  6. 2024Avoiding mode collapse in diffusion models fine-tuned with reinforcement learning
  7. 2026On the Trainability of Masked Diffusion Language Models via Blockwise Locality
  8. 2026BRACE: Anchored Bellman-Residual Correction for Stale Critics in Asynchronous RL