# PoseAdapter: Dual-Stream 2.5D Controllable Image Generation for Complex Multi-Object Scenes

arXiv: https://arxiv.org/abs/2608.15583 · HTML: https://mlsanity.com/paper/2608.15583/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models.
- Builds on 31 of its 45 references that are in the map.
- First posted on arXiv 16 Aug 2026.

## Placed in

- Diffusion models / [Image generation, text-to-image and editing](https://mlsanity.com/diffusion/image-generation-text-to-image-and-editing/)

## Builds on (most cited first, 30 of 31)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow](https://mlsanity.com/paper/2209.03003/) (2022)
- [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023)
- [Zero-1-to-3: Zero-shot One Image to 3D Object](https://mlsanity.com/paper/2303.11328/) (2023)
- [On the Continuity of Rotation Representations in Neural Networks](https://mlsanity.com/paper/1812.07035/) (2018)
- [Reproducible scaling laws for contrastive language-image learning](https://mlsanity.com/paper/2212.07143/) (2022)
- [FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Space](https://mlsanity.com/paper/2506.15742/) (2025)
- [MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generation](https://mlsanity.com/paper/2302.08113/) (2023)
- [All are Worth Words: A ViT Backbone for Diffusion Models](https://mlsanity.com/paper/2209.12152/) (2022)
- [Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimation](https://mlsanity.com/paper/1901.02970/) (2019)
- [BoxDiff: Text-to-Image Synthesis with Training-Free Box-Constrained Diffusion](https://mlsanity.com/paper/2307.10816/) (2023)
- [UniControl: A Unified Diffusion Model for Controllable Visual Generation In the Wild](https://mlsanity.com/paper/2305.11147/) (2023)
- [InstanceDiffusion: Instance-level Control for Image Generation](https://mlsanity.com/paper/2402.03290/) (2024)
- [Dense Text-to-Image Generation with Attention Modulation](https://mlsanity.com/paper/2308.12964/) (2023)
- [Objectron: A Large Scale Dataset of Object-Centric Videos in the Wild with Pose Annotations](https://mlsanity.com/paper/2012.09988/) (2020)
- [LooseControl: Lifting ControlNet for Generalized Depth Conditioning](https://mlsanity.com/paper/2312.03079/) (2023)
- [GeoDiffusion: Text-Prompted Geometric Control for Object Detection Data Generation](https://mlsanity.com/paper/2306.04607/) (2023)
- [Orient Anything: Learning Robust Object Orientation Estimation from Rendering 3D Models](https://mlsanity.com/paper/2412.18605/) (2024)
- [Cityscapes 3D: Dataset and Benchmark for 9 DoF Vehicle Detection](https://mlsanity.com/paper/2006.07864/) (2020)
- [Orient Anything V2: Unifying Orientation and Rotation Understanding](https://mlsanity.com/paper/2601.05573/) (2026)
- [DeBaRA: Denoising-Based 3D Room Arrangement Generation](https://mlsanity.com/paper/2409.18336/) (2024)
- [Build-A-Scene: Interactive 3D Layout Control for Diffusion-Based Image Generation](https://mlsanity.com/paper/2408.14819/) (2024)
- [Compass Control: Multi Object Orientation Control for Text-to-Image Generation](https://mlsanity.com/paper/2504.06752/) (2025)
- [ORIGEN: Zero-Shot 3D Orientation Grounding in Text-to-Image Generation](https://mlsanity.com/paper/2503.22194/) (2025)
- [SceneDesigner: Controllable Multi-Object Image Generation with 9-DoF Pose Manipulation](https://mlsanity.com/paper/2511.16666/) (2025)

## Abstract (arXiv)

While Text-to-Image (T2I) diffusion models have achieved remarkable success, precise spatial and orientational control in multi-object scenes remains a persistent challenge. Existing methods either rely on computationally expensive dense 3D maps or suffer from severe attribute leakage and "cut-and-paste" artifacts. To address these limitations, we propose PoseAdapter, a lightweight framework for high-fidelity 2.5D controllable image generation. Instead of dense spatial maps, it establishes precise spatial-angular anchors using an efficient condition layout: individual object captions, 2D bounding boxes, and 3D angles. To resolve the generative trade-off between strict instance isolation and global coherence, we introduce a Context-Aware Dual-Stream Representation. By injecting local object tokens and relation-enriched scene tokens into the visual stream of modern MM-DiT architectures via parallel masked and unmasked pathways, PoseAdapter eliminates attribute leakage while preserving natural inter-object relationships and scene-level coherence. To support this paradigm, we construct OrientLayout, a high-quality dataset featuring standardized 2.5D annotations and instance-level decoupled semantics. Extensive experiments demonstrate that PoseAdapter outperforms state-of-the-art baselines in spatial accuracy, orientational precision, and multi-object visual fidelity. Code and dataset will be available at https://github.com/cyf23/PoseAdapter.