mlsanityData to 14 Oct 2026

Self-Verifier Capture in Unified Visual Generators: Diagnosing and Repairing Closed-Loop Self-Reward

arXiv 2610.16500 ↗First posted 12 Oct 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Image-space reward requires decoding and a separate judge; unified visual generators can instead reuse their understanding pathway for decode-free, self-supplied reward. Optimizing against this verifier closes the self-reward loop; shared trainable weights let one update move generation and scoring even with the reward gradient stopped. We call the resulting failure self-verifier capture: self-reward rises while independently measured quality falls. Matched semantic/denoising by shared/frozen cells on Scale-RAE isolate two routes: the policy exploits on-manifold blind spots of a frozen semantic judge, while shared updates additionally rewrite the judge. A frozen judge also becomes stale, and repeated self-refresh yields progressively weaker recoveries. The diagnosis yields a two-part repair: locking the warmed reward read blocks rewriting, while sparse, parameter-disjoint external anchors counter residual exploitation. Unanchored rollouts retain semantic reward across multiple latent noise levels, providing dense latent reward that terminal image-space judging cannot supply, while a drift-triggered controller allocates anchors to high-drift rollouts and propagates their calibration within each group. At matched budgets, the controller improves on fixed-dose anchoring, increasingly so as coverage tightens. Both the unanchored failure and the repair transfer to the higher-performing, architecturally distinct Show-o2. Across both bases, the complete method significantly outperforms a construct-matched every-rollout judge on GenEval and DPG-Bench while externally scoring only 1/16 of rollouts. At 1/32 coverage, it scores higher in all four comparisons and remains jointly non-inferior.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 19 of its 47 references that are in the map. First posted on arXiv 12 Oct 2026.

Builds on

19 of 47 references.

References · 19

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  3. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
  4. 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,188
  5. 2023Training Diffusion Models with Reinforcement Learningcited by 982
  6. 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignmentcited by 972
  7. 2024LLM Evaluators Recognize and Favor Their Own Generationscited by 686
  8. 2025Flow-GRPO: Training Flow Matching Models via Online RLcited by 661
  9. 2024ELLA: Equip Diffusion Models with LLM for Enhanced Semantic Alignmentcited by 475
  10. 2025Show-o2: Improved Native Unified Multimodal Modelscited by 250
  11. 2019Reward Tampering Problems and Solutions in Reinforcement Learning: A Causal Influence Diagram Perspectivecited by 108
  12. 2026Scaling Text-to-Image Diffusion Transformers with Representation Autoencoderscited by 53
7 more
  1. 2025SRUM: Fine-Grained Self-Rewarding for Unified Multimodal Modelscited by 22
  2. 2025Reward Model Overoptimisation in Iterated RLHFcited by 12
  3. 2026KVPO: ODE-Native GRPO for Autoregressive Video Alignment via KV Semantic Explorationcited by 11
  4. 2026LatentUM: Unleashing the Potential of Interleaved Cross-Modal Reasoning via a Latent-Space Unified Modelcited by 8
  5. 2025VA-π: Variational Policy Alignment for Pixel-Aware Autoregressive Generationcited by 7
  6. 2025Breaking the Self-Confirming Loop: Diagnosing and Mitigating Systemic Reward Bias in Self-Rewarding RLcited by 3
  7. 2026Improving Text-to-Image Generation with Intrinsic Self-Confidence Rewardscited by 2

Built on by

Not cited yet.