mlsanityData to 7 Oct 2026

Improving Text-to-Image Generation with Intrinsic Self-Confidence Rewards

arXiv 2603.00918 ↗First posted 1 Mar 2026cs.CV cs.AIcited by 1

On the map: LLM alignment / RLHF / preference optimisation → Diffusion and multimodal model alignment

Abstract (arXiv)

Text-to-image generation powers content creation across design, media, and data augmentation. Post-training of text-to-image generative models is a promising path to improve human preference alignment, factuality, and aesthetics. We introduce SOLACE (Self-Originating LAtent Confidence Estimation), a post-training framework that replaces external reward supervision with an internal self-confidence signal: we re-noise the model's own outputs and measure how accurately it recovers the injected noise, treating low reconstruction error as high self-confidence. SOLACE converts this intrinsic signal into scalar rewards for reinforcement learning, requiring no external reward models, annotators, or preference data. By reinforcing high-confidence generations, SOLACE delivers consistent gains in compositional generation, text rendering, and text-image alignment. Integrating SOLACE with external rewards yields complementary improvements while alleviating reward hacking.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Diffusion and multimodal model alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 87 of its 102 references that are in the map. First posted on arXiv 1 Mar 2026.

Builds on

87 of 102 references.

Closest ancestors · 13

Same direction, newest first

  1. 2025Unified Reward Model for Multimodal Understanding and Generation7 Mar
  2. 2025Score as Action: Fine-Tuning Diffusion Generative Models by Continuous-time Reinforcement Learning3 Feb
  3. 2025Improving Video Generation with Human Feedback23 Jan
  4. 2024OnlineVPO: Align Video Diffusion Model with Online Video-Centric Preference Optimization19 Dec
  5. 2024VideoDPO: Omni-Preference Alignment for Video Diffusion Generation18 Dec
  6. 2024Improving Dynamic Object Interactions in Text-to-Video Generation with AI Feedback3 Dec
  7. 2024Video Diffusion Alignment via Reward Gradients11 Jul
  8. 2024Aesthetic Post-Training Diffusion Models from Generic Preferences with Step-by-step Preference Optimization6 Jun
  9. 2024Self-Play Fine-Tuning of Diffusion Models for Text-to-Image Generation15 Feb
  10. 2023Using Human Feedback to Fine-tune Diffusion Models without Any Reward Model22 Nov
  11. 2023Diffusion Model Alignment Using Direct Preference Optimization21 Nov
  12. 2023Directly Fine-Tuning Diffusion Models on Differentiable Rewards29 Sep
1 more
  1. 2023Aligning Text-to-Image Models using Human Feedback23 Feb

Other references · 74

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2017Proximal Policy Optimization Algorithmscited by 14,441
  5. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  6. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  7. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  8. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  9. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  10. 2022Scalable Diffusion Models with Transformerscited by 5,931
  11. 2022Flow Matching for Generative Modelingcited by 5,426
  12. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
38 more
  1. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  2. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
  3. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
  4. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  5. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
  6. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,536
  7. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,755
  8. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,597
  9. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,418
  10. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,287
  11. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,207
  12. 2022Scaling Laws for Reward Model Overoptimizationcited by 1,056
  13. 2025FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Spacecited by 960
  14. 2023Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generationcited by 950
  15. 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignmentcited by 946
  16. 2023Training Diffusion Models with Reinforcement Learningcited by 939
  17. 2023MVDream: Multi-view Diffusion for 3D Generationcited by 928
  18. 2022STaR: Bootstrapping Reasoning With Reasoningcited by 895
  19. 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 854
  20. 2023Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesiscited by 847
  21. 2024Autoregressive Model Beats Diffusion: Llama for Scalable Image Generationcited by 760
  22. 2019Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learningcited by 750
  23. 2023RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignmentcited by 672
  24. 2024Self-Rewarding Language Modelscited by 654
  25. 2023ModelScope Text-to-Video Technical Reportcited by 651
  26. 2023Muse: Text-To-Image Generation via Masked Generative Transformerscited by 640
  27. 2025Flow-GRPO: Training Flow Matching Models via Online RLcited by 605
  28. 2024Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Modelscited by 552
  29. 2024LTX-Video: Realtime Video Latent Diffusioncited by 549
  30. 2022All are Worth Words: A ViT Backbone for Diffusion Modelscited by 498
  31. 2022MagicVideo: Efficient Video Generation With Latent Diffusion Modelscited by 463
  32. 2024OmniGen: Unified Image Generationcited by 388
  33. 2024PixArt-Sigma: Weak-to-Strong Training of Diffusion Transformer for 4K Text-to-Image Generationcited by 326
  34. 2025Absolute Zero: Reinforced Self-play Reasoning with Zero Datacited by 323
  35. 2023Emu Edit: Precise Image Editing via Recognition and Generation Taskscited by 321
  36. 2024An Image is Worth 32 Tokens for Reconstruction and Generationcited by 295
  37. 2023Aligning Text-to-Image Diffusion Models with Reward Backpropagationcited by 284
  38. 2025TTRL: Test-Time Reinforcement Learningcited by 253

50 most cited of 74 All 74 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Reward Hacking in the Era of Large Models: Mechanisms, Emergent Misalignment, Challengescited by 30

Similar, not cited

Close in wording, no citation either way

  1. 2026Post-Training Frontier Text-to-Image Models by Composing Preference and Rubric Rewards
  2. 2025Self-Rewarding Large Vision-Language Models for Optimizing Prompts in Text-to-Image Generation
  3. 2023Rich Human Feedback for Text-to-Image Generation
  4. 2025Enhancing Reward Models for High-quality Image Generation: Beyond Text-Image Alignment
  5. 2026Trust Your Critic: Robust Reward Modeling and Reinforcement Learning for Faithful Image Editing and Generation
  6. 2024Multimodal Large Language Model is a Human-Aligned Annotator for Text-to-Image Generation
  7. 2025InstructEngine: Instruction-driven Text-to-Image Alignment
  8. 2024Diff-Instruct++: Training One-step Text-to-image Generator Model to Align with Human Preferences