mlsanityData to 14 Oct 2026

COLLAR: Cascaded Object-Level Latent Refinement for High-Fidelity Conditional Generation

arXiv 2606.00954 ↗First posted 31 May 2026cs.CVcited by 0

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

Achieving high-fidelity object-level control in Diffusion Transformers remains a significant challenge despite the introduction of structural priors like depth and Canny maps. Current object-level conditional generation methods frequently suffer from visual artifacts and struggle to maintain precise control over objects within small localized regions. To address these limitations, we propose Cascaded Object-Level Latent Refinement (COLLAR), a training-free framework that progressively optimizes object-level features via the Field-of-View (FoV) expansion. First, we propose the Cross-Scale Semantic Alignment (CSSA) module to address spatial-semantic gaps by injecting object-level features into extended-FoV branches via attention mechanisms. To further optimize these features, the Cyclic Feature Injection (CFI) module introduces a reciprocal background feedback mechanism. It leverages a frequency-based adaptive strategy to selectively update the global backbone with context-aligned local information. Finally, the extended-FoV branch serves as a hub for feature optimization, ensuring that object-level features are integrated into the global generation process without compromising final image quality. Extensive experiments on the COCO-MIG and COCO-POS benchmarks demonstrate that our approach consistently outperforms state-of-the-art methods across semantic alignment, image quality, and spatial fidelity.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 24 of its 37 references that are in the map. First posted on arXiv 31 May 2026.

Builds on

24 of 37 references.

Closest ancestors · 12

Same direction, newest first

  1. 2025InstanceAssemble: Layout-Aware Image Generation via Instance Assembling Attention20 Sep
  2. 2024CreatiLayout: Siamese Multimodal Diffusion Transformer for Creative Layout-to-Image Generation5 Dec
  3. 2024Training-free Regional Prompting for Diffusion Transformers4 Nov
  4. 2024GrounDiT: Grounding Diffusion Transformers via Noisy Patch Transplantation27 Oct
  5. 2024HiCo: Hierarchical Controllable Diffusion Model for Layout-to-image Generation18 Oct
  6. 2024Enhancing Prompt Following with Visual Control Through Training-Free Mask-Guided Diffusion23 Apr
  7. 2024NoiseCollage: A Layout-Aware Text-to-Image Diffusion Model Based on Noise Cropping and Merging6 Mar
  8. 2024InstanceDiffusion: Instance-level Control for Image Generation5 Feb
  9. 2023Cocktail: Mixing Multi-Modality Controls for Text-Conditional Image Generation1 Jun
  10. 2023LayoutDiffusion: Controllable Diffusion Model for Layout-to-image Generation30 Mar
  11. 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models16 Feb
  12. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding23 May

Other references · 12

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  4. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
  5. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  6. 2023Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generationcited by 977
  7. 2025SAM 3: Segment Anything with Conceptscited by 906
  8. 2016Generative Adversarial Text to Image Synthesiscited by 735
  9. 2025HPSv3: Towards Wide-Spectrum Human Preference Scorecited by 211
  10. 2023MotionEditor: Editing Video Motion via Content-Aware Diffusioncited by 57
  11. 2024Cropper: Vision-Language Model for Image Cropping through In-Context Learningcited by 11
  12. 2025Seg2Any: Open-set Segmentation-Mask-to-Image Generation with Precise Shape and Semantic Controlcited by 10

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Beyond Point-Wise Matching: Structural Representation Alignment for Accelerating Diffusion Transformers
  2. 2026A Hidden Semantic Bottleneck in Conditional Embeddings of Diffusion Transformers
  3. 2026Training-Free Representation Guidance for Diffusion Models with a Representation Alignment Projector
  4. 2026A training-free framework for high-fidelity appearance transfer via diffusion transformers
  5. 2024Improving Text-guided Object Inpainting with Semantic Pre-inpainting
  6. 2024ST-LDM: A Universal Framework for Text-Grounded Object Generation in Real Images
  7. 2024MOVIS: Enhancing Multi-Object Novel View Synthesis for Indoor Scenes
  8. 2024A Simple Approach to Unifying Diffusion-based Conditional Generation