mlsanityData to 7 Oct 2026

Blackboard Intelligence Can Surpass Autoregressive on Globally Constrained Problems

arXiv 2609.38806 ↗First posted 30 Sep 2026cs.LG cs.CLcited by 0

On the map: Diffusion models → Discrete and text diffusion

Abstract (arXiv)

Next-token prediction has driven remarkable progress in large language models, yet a growing body of evidence suggests that they can struggle on problems governed by complex global constraints. In this work, we focus on this regime and ask whether some of these limitations arise from the inference interface induced by next-token prediction itself. We study this question through blackboard intelligence: an inference-time perspective in which a model works on a fixed, revisable canvas and searches over candidate solution states rather than committing to a causal, left-to-right trajectory. We instantiate this idea with diffusion language models, whose any-order prediction interface naturally exposes predictions over partially filled solution states. Our key observation is that mean confidence, a simple model-internal quantity available from the standard masked diffusion objective, provides a useful proxy for global coherence and can guide inference-time search and revision. Empirically, across ZebraLogic, Nurse Rostering, and Job-Shop Scheduling, Blackboard consistently improves inference while holding the fine-tuned LLaDA-8B-Instruct checkpoint fixed and substantially outperforms same-scale autoregressive baselines, reaching 90.4% accuracy on ZebraLogic-Hard, 76.4% exact feasibility on Nurse Rostering, and 80.2% optimality on JSSP. Stronger autoregressive search and refinement also fail to close the gap on ZebraLogic-Hard, while Blackboard surpasses tested frontier LLMs there and on JSSP despite their substantially greater scale and strong test-time reasoning. We open-source our codebase at https://github.com/jwoosang1/blackboard-intelligence.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Discrete and text diffusion, one of 14 directions in Diffusion models. Builds on 35 of its 41 references that are in the map. First posted on arXiv 30 Sep 2026.

Builds on

35 of 41 references.

Closest ancestors · 19

Same direction, newest first

  1. 2026Learn from Your Mistakes: Self-Correcting Masked Diffusion Models12 Feb
  2. 2026Improving Diffusion Language Model Decoding through Joint Search in Generation Order and Token Space28 Jan
  3. 2025From Bits to Rounds: Parallel Decoding with Exploration for Diffusion Language Models26 Nov
  4. 2025Lookahead Unmasking Elicits Accurate Decoding in Diffusion Language Models4 Nov
  5. 2025Demystifying MaskGIT Sampler and Beyond: Adaptive Order Selection in Masked Diffusion6 Oct
  6. 2025Fine-Tuning Masked Diffusion for Provable Self-Correction1 Oct
  7. 2025Don't Settle Too Early: Self-Reflective Remasking for Diffusion Language Models28 Sep
  8. 2025Seed Diffusion: A Large-Scale Diffusion Language Model with High-Speed Inference4 Aug
  9. 2025DiffuCoder: Understanding and Improving Masked Diffusion Models for Code Generation25 Jun
  10. 2025Accelerated Sampling from Masked Diffusion Models via Entropy Bounded Unmasking30 May
  11. 2025Remasking Discrete Diffusion Models with Inference-Time Scaling1 Mar
  12. 2025Large Language Diffusion Models14 Feb
7 more
  1. 2025Train for the Worst, Plan for the Best: Understanding Token Ordering in Masked Diffusions10 Feb
  2. 2025Path Planning for Masked Diffusion Model Sampling5 Feb
  3. 2024Beyond Autoregression: Discrete Diffusion for Complex Reasoning and Planning18 Oct
  4. 2024Masked Diffusion Models are Secretly Time-Agnostic Masked Models and Exploit Inaccurate Categorical Sampling4 Sep
  5. 2024Simple and Effective Masked Diffusion Language Models11 Jun
  6. 2024Simplified and Generalized Masked Diffusion for Discrete Data6 Jun
  7. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution25 Oct

Other references · 16

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  4. 2025Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decodingcited by 404
  5. 2025Mercury: Ultra-Fast Language Models Based on Diffusioncited by 225
  6. 2025ZebraLogic: On the Scaling Limits of LLMs for Logical Reasoningcited by 186
  7. 2025Fast-dLLM v2: Efficient Block-Diffusion LLMcited by 128
  8. 2025Enigmata: Scaling Logical Reasoning in Large Language Models with Synthetic Verifiable Puzzlescited by 53
  9. 2025SATBench: Benchmarking LLMs' Logical Reasoning via Automated Puzzle Generation from SAT Formulascited by 30
  10. 2025Roll the dice & look before you leap: Going beyond the creative limits of next-token predictioncited by 28
  11. 2025LoPA: Scaling dLLM Inference via Lookahead Parallel Decodingcited by 17
  12. 2025Starjob: Dataset for LLM-Driven Job Shop Schedulingcited by 12
4 more
  1. 2026Reasoning in a Combinatorial and Constrained World: Benchmarking LLMs on Natural-Language Combinatorial Optimizationcited by 9
  2. 2026ConstraintBench: Benchmarking LLM Constraint Reasoning on Direct Optimizationcited by 6
  3. 2026Evaluating Relational Reasoning in LLMs with RELcited by 2
  4. 2025Interleaved Gibbs Diffusion: Generating Discrete-Continuous Data with Implicit Constraintscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024The pitfalls of next-token prediction
  2. 2026Reasoning with Latent Tokens in Diffusion Language Models
  3. 2026Beyond the Autoregressive Horizon: A Comprehensive Survey of Diffusion Models, World Modelling, and State Space Models for Code
  4. 2026Early Decisions Matter: Proximity Bias and Initial Trajectory Shaping in Non-Autoregressive Diffusion Language Models
  5. 2026ForkLeft: Entropy-First Rollouts for Prefix-Aligned Autoregressive-to-Diffusion Distillation
  6. 2025Beyond Exponential Decay: Rethinking Error Accumulation in Large Language Models
  7. 2024When a language model is optimized for reasoning, does it still show embers of autoregression? An analysis of OpenAI o1
  8. 2026From Interface to Inference: Eliciting Any-Order Inference from Any-Order Models