mlsanityData to 7 Oct 2026

Oracle Noise: Faster Semantic Spherical Alignment for Interpretable Latent Optimization

arXiv 2604.23540 ↗First posted 26 Apr 2026cs.CVcited by 2

On the map: Diffusion models → Image generation, text-to-image and editing

also in 1 more
Abstract (arXiv)

Text-to-image diffusion models have achieved remarkable generative capabilities, yet accurately aligning complex textual prompts with synthesized layouts remains an ongoing challenge. In these models, the initial Gaussian noise acts as a critical structural seed dictating the macroscopic layout. Recent online optimization and search methods attempt to refine this noise to enhance text-image alignment. However, relying on unconstrained Euclidean gradient ascent mathematically inflates the latent norm and destroys the standard Gaussian prior, causing severe visual artifacts like color over-saturation. Furthermore, these methods suffer from inefficient semantic routing and easily fall into the ``reward hacking'' trap of external proxy models. To address these intertwined bottlenecks, we propose Oracle Noise, a zero-shot framework reframing noise initialization as semantic-driven optimization strictly confined to a Riemannian hypersphere. Instead of relying on complex external parsers, we directly identify the most impactful structural words in the prompt to efficiently route optimization energy. By updating the noise strictly along a spherical path, we mathematically preserve the original Gaussian distribution. This geometric constraint eliminates norm inflation and unlocks aggressive step sizes for rapid convergence. Extensive experiments demonstrate that Oracle Noise significantly accelerates semantic alignment and achieves superior aesthetics without black-box models. It completely mitigates Euclidean-induced degradation, establishing state-of-the-art performance across human preference metrics (e.g., HPSv2, ImageReward), semantic alignment (CLIP Score), and sample diversity, all within a strict 2-second optimization budget.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 35 of its 38 references that are in the map. First posted on arXiv 26 Apr 2026.

Builds on

35 of 38 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024InitNO: Boosting Text-to-Image Diffusion Models via Initial Noise Optimization6 Apr
  2. 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignment17 Oct
  3. 2022What the DAAM: Interpreting Stable Diffusion Using Cross Attention10 Oct
  4. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding23 May

Other references · 31

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  5. 2020Denoising Diffusion Implicit Modelscited by 9,655
  6. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  7. 2022Scalable Diffusion Models with Transformerscited by 5,931
  8. 2022Classifier-Free Diffusion Guidancecited by 5,717
  9. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  10. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  11. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
  12. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
19 more
  1. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
  2. 2023Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generationcited by 950
  3. 2022Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translationcited by 897
  4. 2023Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesiscited by 847
  5. 2022The Vendi Score: A Diversity Evaluation Metric for Machine Learningcited by 299
  6. 2025Inference-Time Scaling for Diffusion Models beyond Scaling Denoising Stepscited by 225
  7. 2018Hyperspherical Variational Auto-Encoderscited by 188
  8. 2024ReNO: Enhancing One-step Text-to-Image Models through Reward-based Noise Optimizationcited by 115
  9. 2024Golden Noise for Diffusion Models: A Learning Frameworkcited by 91
  10. 2024Not All Noises Are Created Equally:Diffusion Noise Selection and Optimizationcited by 70
  11. 2024A Noise is Worth Diffusion Guidancecited by 48
  12. 2023Zeroth-Order Optimization Meets Human Feedback: Provable Learning via Ranking Oraclescited by 34
  13. 2024DCTdiff: Intriguing Properties of Image Generative Modeling in the DCT Spacecited by 32
  14. 2024SATO: Stable Text-to-Motion Frameworkcited by 20
  15. 2024The Silent Assistant: NoiseQuery as Implicit Guidance for Goal-Driven Image Generationcited by 14
  16. 2025Physics-Informed Representation Alignment for Sparse Radio-Map Reconstructioncited by 11
  17. 2025CoEmoGen: Towards Semantically-Coherent and Scalable Emotional Image Content Generationcited by 9
  18. 2025POLARIS: Projection-Orthogonal Least Squares for Robust and Adaptive Inversion in Diffusion Modelscited by 8
  19. 2025Harmonizing Geometry and Uncertainty: Diffusion with Hyperspherescited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025OptiPrune: Boosting Prompt-Image Consistency with Attention-Guided Noise and Dynamic Token Selection
  2. 2025CARINOX: Inference-time Scaling with Category-Aware Reward-based Initial Noise Optimization and Exploration
  3. 2024Noise Diffusion for Enhancing Semantic Faithfulness in Text-to-Image Synthesis
  4. 2025Noise Projection: Closing the Prompt-Agnostic Gap Behind Text-to-Image Misalignment in Diffusion Models
  5. 2026Intermediate Text Representation Guided Text-to-Image Generation for Enhancing One-and-Only Alignment
  6. 2026Alignment-Guided Score Matching for Text-to-Image Alignment in Diffusion Models
  7. 2025InfSplign: Inference-Time Spatial Alignment of Text-to-Image Diffusion Models
  8. 2025Test-Time Alignment of Text-to-Image Diffusion Models via Null-Text Embedding Optimisation