mlsanityData to 14 Oct 2026

Evidence-Aligned Local Composition of Discrete Experts for Sequence Restoration

arXiv 2609.05801 ↗First posted 5 Sep 2026cs.AI cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

A document modeled as a discrete sequence of tokens can be thought of as being generated from a composition of texts from different domains; a README file, for example, moves between prose, code, and configuration. When such a document is corrupted and only frozen domain experts are available, restoring it requires deciding both what is missing and which expert to trust at each position, at test time and without region labels or a trained router. We introduce evidence-aligned local composition, which infers a soft, position-wise weighting over the experts from the marginal evidence of the corrupted observation under a given corruption model, estimating the evidence from the experts' own denoising losses and smoothing the weights across positions. Because the weighting is soft, it recovers a mixture when the true composition is mixed and concentrates on one expert when that suffices. Across a categorical simulator, byte-level experts, and experts fine-tuned from a 1.3B discrete flow-matching model, the inferred weights track the true regions at 0.85 field accuracy on naturally mixed scientific documents, and at 0.98 on constructed mixtures whose regions are lexically disjoint. Restoration improves over a single global weight when the experts are genuinely distinct and reduces to it when they converge, tracking a measure of expert separation.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 30 of its 39 references that are in the map. First posted on arXiv 5 Sep 2026.

Builds on

30 of 39 references.

References · 30

  1. 2022Flow Matching for Generative Modelingcited by 5,702
  2. 2016Pointer Sentinel Mixture Modelscited by 3,096
  3. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,719
  4. 2022Diffusion Posterior Sampling for General Noisy Inverse Problemscited by 1,416
  5. 2022Denoising Diffusion Restoration Modelscited by 963
  6. 2024Simple and Effective Masked Diffusion Language Modelscited by 848
  7. 2023Discrete Diffusion Modeling by Estimating the Ratios of the Data Distributioncited by 745
  8. 2019CodeSearchNet Challenge: Evaluating the State of Semantic Code Searchcited by 654
  9. 2024Simplified and Generalized Masked Diffusion for Discrete Datacited by 626
  10. 2022Compositional Visual Generation with Composable Diffusion Modelscited by 624
  11. 2023LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusioncited by 564
  12. 2022A Continuous Time Framework for Discrete Denoising Modelscited by 451
18 more
  1. 2024Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Datacited by 381
  2. 2024Discrete Flow Matchingcited by 317
  3. 2024Scaling Diffusion Language Models via Adaptation from Autoregressive Modelscited by 274
  4. 2023Reduce, Reuse, Recycle: Compositional Generation with Energy-Based Diffusion Models and MCMCcited by 220
  5. 2021Autoregressive Diffusion Modelscited by 210
  6. 2024Scaling up Masked Diffusion Models on Textcited by 199
  7. 2024Simple Guidance Mechanisms for Discrete Diffusion Modelscited by 155
  8. 2021DEMix Layers: Disentangling Domains for Modular Language Modelingcited by 132
  9. 2024Unlocking Guidance for Discrete State-Space Diffusion and Flow Modelscited by 129
  10. 2024Knowledge Fusion of Large Language Modelscited by 115
  11. 2024Branch-Train-MiX: Mixing Expert LLMs into a Mixture-of-Experts LLMcited by 109
  12. 2013A Deep and Tractable Density Estimatorcited by 68
  13. 2024Pack of LLMs: Model Fusion at Test-Time via Perplexity Optimizationcited by 56
  14. 2024The Superposition of Diffusion Models Using the Ito Density Estimatorcited by 36
  15. 2023Compositional Sculpting of Iterative Generative Processescited by 27
  16. 2025Test-Time Steering for Lossless Text Compression via Weighted Product of Expertscited by 3
  17. 2026Sample-efficient evidence estimation of score based priors for model selectioncited by 3
  18. 2026Optimizing Diffusion Priors in Image Reconstruction from a Single Observationcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026RaV-IDP: A Reconstruction-as-Validation Framework for Faithful Intelligent Document Processing
  2. 2021Beyond In-Place Corruption: Insertion and Deletion In Denoising Probabilistic Models
  3. 2026Expert Selections In MoE Models Reveal (Almost) As Much As Text
  4. 2026SciDocBench: A Workflow-Centered Benchmark and Data Pipeline for Scientific Document Understanding
  5. 2025As If We've Met Before: LLMs Exhibit Certainty in Recognizing Seen Files
  6. 2026AtomCite: Verification and Correction of Supplied Page-Level Citations in Multi-Page Documents
  7. 2026Expert-Aware Causal Tracing of Factual Recall in Sparse MoE Language Models
  8. 2026TokenPrint: A Calibrated Token-Space Fingerprint for Language-Model Provenance