mlsanityData to 14 Oct 2026

PoseAdapter: Dual-Stream 2.5D Controllable Image Generation for Complex Multi-Object Scenes

arXiv 2608.15583 ↗First posted 16 Aug 2026cs.CVcited by 0

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

While Text-to-Image (T2I) diffusion models have achieved remarkable success, precise spatial and orientational control in multi-object scenes remains a persistent challenge. Existing methods either rely on computationally expensive dense 3D maps or suffer from severe attribute leakage and "cut-and-paste" artifacts. To address these limitations, we propose PoseAdapter, a lightweight framework for high-fidelity 2.5D controllable image generation. Instead of dense spatial maps, it establishes precise spatial-angular anchors using an efficient condition layout: individual object captions, 2D bounding boxes, and 3D angles. To resolve the generative trade-off between strict instance isolation and global coherence, we introduce a Context-Aware Dual-Stream Representation. By injecting local object tokens and relation-enriched scene tokens into the visual stream of modern MM-DiT architectures via parallel masked and unmasked pathways, PoseAdapter eliminates attribute leakage while preserving natural inter-object relationships and scene-level coherence. To support this paradigm, we construct OrientLayout, a high-quality dataset featuring standardized 2.5D annotations and instance-level decoupled semantics. Extensive experiments demonstrate that PoseAdapter outperforms state-of-the-art baselines in spatial accuracy, orientational precision, and multi-object visual fidelity. Code and dataset will be available at https://github.com/cyf23/PoseAdapter.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 31 of its 45 references that are in the map. First posted on arXiv 16 Aug 2026.

Builds on

31 of 45 references.

Closest ancestors · 10

Same direction, newest first

  1. 2025FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Space17 Jun
  2. 2025Compass Control: Multi Object Orientation Control for Text-to-Image Generation9 Apr
  3. 2024Build-A-Scene: Interactive 3D Layout Control for Diffusion-Based Image Generation27 Aug
  4. 2024InstanceDiffusion: Instance-level Control for Image Generation5 Feb
  5. 2023LooseControl: Lifting ControlNet for Generalized Depth Conditioning5 Dec
  6. 2023Dense Text-to-Image Generation with Attention Modulation24 Aug
  7. 2023BoxDiff: Text-to-Image Synthesis with Training-Free Box-Constrained Diffusion20 Jul
  8. 2023GeoDiffusion: Text-Prompted Geometric Control for Object Detection Data Generation7 Jun
  9. 2023UniControl: A Unified Diffusion Model for Controllable Visual Generation In the Wild18 May
  10. 2023MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generation16 Feb

Other references · 21

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  4. 2022Classifier-Free Diffusion Guidancecited by 5,847
  5. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
  6. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
  7. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
  8. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  9. 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,489
  10. 2018On the Continuity of Rotation Representations in Neural Networkscited by 1,214
  11. 2022Reproducible scaling laws for contrastive language-image learningcited by 1,173
  12. 2022All are Worth Words: A ViT Backbone for Diffusion Modelscited by 502
9 more
  1. 2019Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimationcited by 495
  2. 2020Objectron: A Large Scale Dataset of Object-Centric Videos in the Wild with Pose Annotationscited by 155
  3. 2024Orient Anything: Learning Robust Object Orientation Estimation from Rendering 3D Modelscited by 45
  4. 2020Cityscapes 3D: Dataset and Benchmark for 9 DoF Vehicle Detectioncited by 39
  5. 2026Orient Anything V2: Unifying Orientation and Rotation Understandingcited by 23
  6. 2024DeBaRA: Denoising-Based 3D Room Arrangement Generationcited by 20
  7. 2025ORIGEN: Zero-Shot 3D Orientation Grounding in Text-to-Image Generationcited by 10
  8. 2025SceneDesigner: Controllable Multi-Object Image Generation with 9-DoF Pose Manipulationcited by 9
  9. 2025LACONIC: A 3D Layout Adapter for Controllable Image Creationcited by 6

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026TrioPose: Native Triple-Stream Diffusion Transformers for Pose-Guided Text-to-Image Generation
  2. 2025Dual Recursive Feedback on Generation and Appearance Latents for Pose-Robust Text-to-Image Diffusion
  3. 2024Neural Assets: 3D-Aware Multi-Object Scene Synthesis with Image Diffusion Models
  4. 2024From Text to Pose to Image: Improving Diffusion Model Control and Quality
  5. 2023DreamComposer: Controllable 3D Object Generation via Multi-View Conditions
  6. 2025Stitch: Training-Free Position Control in Multimodal Diffusion Transformers
  7. 20243D-SceneDreamer: Text-Driven 3D-Consistent Scene Generation
  8. 2025Setting the Stage: Text-Driven Scene-Consistent Image Generation