mlsanityData to 7 Oct 2026

RADIANCE: Relative Adaptive Denoising with IP-Adapter for Novel Concept Enhancement

arXiv 2607.05088 ↗First posted 6 Jul 2026cs.CVcited by 0

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

Text-to-image (T2I) diffusion models have achieved striking progress but still struggle to synthesize rare concepts involving unusual attribute-object pairings, often resulting in concept omission or semantic drift where a dominant entity overwhelms the generation. Tracing these failures to a lack of compositional balance during the denoising trajectory, we propose RADIANCE, a training-free framework that treats inference as a closed-loop feedback process. RADIANCE augments pretrained backbones with three modular components: (1) a Compositional Similarity Monitor (CSM) that tracks the emergence of objects and attributes in intermediate latents via CLIP-based feedback; (2) a Bidirectional Scale Controller (BSC) that applies a reactive "restoring force" using positive and negative IP-Adapter scales to rebalance biased trajectories; and (3) a Feedback Guidance Scheduler (FGS) that coordinates these updates across timesteps without additional training. We further extend the framework to multi-object prompts via Delayed Adapter Activation (DAA) and Layer-wise Alternating Guidance (LAG) to prevent premature concept fusion. By overlapping monitoring and denoising through pipelined execution, RADIANCE maintains competitive latency while significantly enhancing the per-sample success rate and effective throughput. Experiments on RareBench and T2I-CompBench demonstrate that RADIANCE consistently enhances compositional alignment and perceptual quality over state-of-the-art baselines.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 35 of its 39 references that are in the map. First posted on arXiv 6 Jul 2026.

Builds on

35 of 39 references.

Closest ancestors · 15

Same direction, newest first

  1. 2025Leveraging Semantic Attribute Binding for Free-Lunch Color Control in Diffusion Models12 Mar
  2. 2024Rare-to-Frequent: Unlocking Compositional Generation Power of Diffusion Models on Rare Concepts with LLM Guidance29 Oct
  3. 2024ColorPeel: Color Prompt Learning with Diffusion Models via Color and Shape Disentanglement9 Jul
  4. 2024Improving Compositional Attribute Binding in Text-to-Image Generative Models via Enhanced Text Embeddings12 Jun
  5. 2024InitNO: Boosting Text-to-Image Diffusion Models via Initial Noise Optimization6 Apr
  6. 2024ELLA: Equip Diffusion Models with LLM for Enhanced Semantic Alignment8 Mar
  7. 2024Mastering Text-to-Image Diffusion: Recaptioning, Planning, and Generating with Multimodal LLMs22 Jan
  8. 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models13 Aug
  9. 2023Linguistic Binding in Diffusion Models: Enhancing Attribute Correspondence through Attention Map Alignment15 Jun
  10. 2023LLM-grounded Diffusion: Enhancing Prompt Understanding of Text-to-Image Diffusion Models with Large Language Models23 May
  11. 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models16 Feb
  12. 2023Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Models31 Jan
3 more
  1. 2023GLIGEN: Open-Set Grounded Text-to-Image Generation17 Jan
  2. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding23 May
  3. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models20 Dec

Other references · 20

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  6. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  7. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  8. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  9. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
  10. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
  11. 2022Diffusion Models: A Comprehensive Survey of Methods and Applicationscited by 1,107
  12. 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 854
8 more
  1. 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 787
  2. 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 734
  3. 2021CogView: Mastering Text-to-Image Generation via Transformerscited by 691
  4. 2023Generative Multimodal Models are In-Context Learnerscited by 446
  5. 2023Composer: Creative and Controllable Image Synthesis with Composable Conditionscited by 303
  6. 2023T2I-CompBench++: An Enhanced and Comprehensive Benchmark for Compositional Text-to-image Generationcited by 253
  7. 2023Improving Automatic VQA Evaluation Using Large Language Modelscited by 37
  8. 2025TF-TI2I: Training-Free Text-and-Image-to-Image Generation via Multi-Modal Implicit-Context Learning in Text-to-Image Modelscited by 6

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Interpreting Object-Dependent Concept Brittleness in Text-to-Image Diffusion Models
  2. 2026Rectify Then Diffuse: Disentangling Concepts Before Denoising Trajectory Unfolds
  3. 2026Concept Guidance: Precise, Training-Free Latent Control for Text-to-Image Generation
  4. 2026CoBind: Stage-Aware Compositional Binding for Training-Free Text-to-Image Generation
  5. 2026Diff-Aid: Inference-time Adaptive Interaction Denoising for Rectified Text-to-Image Generation
  6. 2026Adaptive Auxiliary Prompt Blending for Target-Faithful Diffusion Generation
  7. 2026Compositional Concept Erasure in Text-to-Image Diffusion Models via Hierarchically Grounded Semantic Surgery
  8. 2026Panoptic Scene Program Diffusion Transformer