RefCompose: Multi-Reference Image Generation via LoRA-Conditioned Diffusion
Not on the map. Why?
Abstract (arXiv)
Filmmakers and visual artists routinely need to compose multiple references, actors, locations, props, cultural elements, into a single coherent shot, but existing tools either fail to scale past a handful of references or destroy fine grained subject identity in the process, since per reference tokenization scales memory linearly with reference count N and generated content often departs from the given references rather than reproducing them. We propose RefCompose, a pixel space compositional conditioning framework that decouples where things go from what they look like, via a single fixed resolution reference canvas that keeps conditioning size constant regardless of reference count. Spatial layout is induced at inference time from a frozen diffusion transformer and extracted via Grounding DINO, requiring no LLM or dedicated layout model, while dual stream LoRA adapters inject a layout derived depth map and the encoded canvas through separate low rank streams, disentangling geometric scaffolding from localized appearance. On the Dense Layout protocol, RefCompose consistently outperforms layout based and state of the art multi reference baselines on color, texture, shape, spatial accuracy, and identity/content preservation at higher reference counts, all with constant inference memory, making it a practical building block for multi subject cinematic composition at production scale.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 29 of its 31 references that are in the map. First posted on arXiv 26 Sep 2026.
Builds on
29 of 31 references.
References · 29
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
- 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
- 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
- 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
- 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
- 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
- 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Modelscited by 1,439
- 2022Null-text Inversion for Editing Real Images using Guided Diffusion Modelscited by 1,168
- 2022Multi-Concept Customization of Text-to-Image Diffusioncited by 1,099
- 2023Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generationcited by 950
- 2022Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translationcited by 897
17 more
- 2023GLIGEN: Open-Set Grounded Text-to-Image Generationcited by 774
- 2025Depth Anything 3: Recovering the Visual Space from Any Viewscited by 626
- 2022Compositional Visual Generation with Composable Diffusion Modelscited by 619
- 2024InstantID: Zero-shot Identity-Preserving Generation in Secondscited by 431
- 2024OmniGen: Unified Image Generationcited by 388
- 2023LayoutGPT: Compositional Visual Planning and Generation with Large Language Modelscited by 337
- 2023PhotoMaker: Customizing Realistic Human Photos via Stacked ID Embeddingcited by 324
- 2023LayoutDiffusion: Controllable Diffusion Model for Layout-to-image Generationcited by 258
- 2023LLM-grounded Diffusion: Enhancing Prompt Understanding of Text-to-Image Diffusion Models with Large Language Modelscited by 242
- 2024Mixture of LoRA Expertscited by 179
- 2025Less-to-More Generalization: Unlocking More Controllability by In-Context Generationcited by 153
- 2024MS-Diffusion: Multi-subject Zero-shot Image Personalization with Layout Guidancecited by 142
- 2025EasyControl: Adding Efficient and Flexible Control for Diffusion Transformercited by 118
- 2024CreatiLayout: Siamese Multimodal Diffusion Transformer for Creative Layout-to-Image Generationcited by 58
- 2025XVerse: Consistent Multi-Subject Control of Identity and Semantic Attributes via DiT Modulationcited by 55
- 2025InstanceAssemble: Layout-Aware Image Generation via Instance Assembling Attentioncited by 9
- 2025Canvas-to-Image: Compositional Image Generation with Multimodal Controlscited by 5
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026RefRoute: Decoupling Conditioning Cost from References via Compact Residual Conditioning and Spatial Routing
- 2025OmniRefiner: Reinforcement-Guided Local Diffusion Refinement
- 2023Reference-based Image Composition with Sketch via Structure-aware Diffusion Model
- 2023Reference-based Painterly Inpainting via Diffusion: Crossing the Wild Reference Domain Gap
- 2025MultiRef: Controllable Image Generation with Multiple Visual References
- 2025Refaccade: Editing Object with Given Reference Texture
- 2025A Two-Stage System for Layout-Controlled Image Generation using Large Language Models and Diffusion Models
- 2025PSDiffusion: Harmonized Multi-Layer Image Generation via Layout and Appearance Alignment