TrioPose: Native Triple-Stream Diffusion Transformers for Pose-Guided Text-to-Image Generation
On the map: Diffusion models → Image generation, text-to-image and editing
Abstract (arXiv)
Pose-guided text-to-image generation often suffers from limb distortions and feature crosstalk in complex multi-person scenarios. While existing UNet-based adapters struggle with long-range spatial dependencies, emerging Multimodal Diffusion Transformers (MM-DiTs) offer superior global modeling. However, naive signal concatenation in MM-DiTs severely disrupts pre-trained latent distributions. To address this, we propose TrioPose, a native pose-driven framework built upon the SD3.5M architecture. Specifically, we introduce a Triple-Stream Pose-Aware DiT (TSPA-DiT) that treats pose as an independent modality. It employs layer-wise activation and zero-initialized dual-residual injection to smoothly enforce geometric constraints while preserving pre-trained latent stability. To resolve severe multi-instance occlusions, we design a Learnable Relational Bias Mask that categorizes topological connectivity into fine-grained physical states, mapping them into continuous attention soft constraints to effectively decouple inter-instance interference. Furthermore, a Pose-Guided Spatial Loss Weighting strategy modulates the native diffusion objective using heatmap-derived error maps, focusing anatomical supervision strictly on distortion-prone regions. Extensive experiments demonstrate that TrioPose achieves state-of-the-art performance across challenging benchmarks, including Human-Art, CrowdPose, and OCHuman. Notably, it attains an AP of 64.33 on Human-Art, representing a 30% improvement over prior arts, while setting new standards for visual fidelity and text-image semantic alignment in complex multi-human generation.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 33 of its 42 references that are in the map. First posted on arXiv 5 Jun 2026.
Builds on
33 of 42 references.
Closest ancestors · 7
Same direction, newest first
- 2025PixelPonder: Dynamic Patch Adaptation for Enhanced Multi-Conditional Text-to-Image Generation9 Mar
- 2023Uni-ControlNet: All-in-One Control to Text-to-Image Diffusion Models25 May
- 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models16 Feb
- 2023Adding Conditional Control to Text-to-Image Diffusion Models10 Feb
- 2023GLIGEN: Open-Set Grounded Text-to-Image Generation17 Jan
- 2022Synthesizing Coherent Story with Auto-Regressive Latent Diffusion Models20 Nov
- 2022Hierarchical Text-Conditional Image Generation with CLIP Latents13 Apr
Other references · 26
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2020Denoising Diffusion Probabilistic Modelscited by 20,851
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
- 2014Microsoft COCO: Common Objects in Contextcited by 17,498
- 2022Scalable Diffusion Models with Transformerscited by 5,931
- 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
- 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
- 2025Qwen3-VL Technical Reportcited by 2,534
- 2018Demystifying MMD GANscited by 1,268
- 2024Champ: Controllable and Consistent Human Image Animation with 3D Parametric Guidancecited by 271
- 2017Pose Guided Person Image Generationcited by 251
- 2018CrowdPose: Efficient Crowded Scenes Pose Estimation and A New Benchmarkcited by 214
14 more
- 2020Controllable Person Image Synthesis with Attribute-Decomposed GANcited by 150
- 2019Progressive Pose Attention Transfer for Person Image Generationcited by 138
- 2018Pose2Seg: Detection Free Human Instance Segmentationcited by 107
- 2023HumanSD: A Native Skeleton-Guided Diffusion Model for Human Image Generationcited by 104
- 2023Human-Art: A Versatile Human-Centric Dataset Bridging Natural and Artificial Scenescited by 68
- 2022ViTPose++: Vision Transformer for Generic Body Pose Estimationcited by 65
- 2021PISE: Person Image Synthesis and Editing with Decoupled GANcited by 62
- 2017Be Your Own Prada: Fashion Synthesis with Structural Coherencecited by 49
- 2021Learning Semantic Person Image Generation by Region-Adaptive Normalizationcited by 44
- 2024GRPose: Learning Graph Relations for Human Image Generation with Pose Priorscited by 14
- 2022KPE: Keypoint Pose Encoding for Transformer-based Image Generationcited by 12
- 2024Stable-Pose: Leveraging Transformers for Pose-Guided Text-to-Image Generationcited by 12
- 2025Rethink Sparse Signals for Pose-guided Text-to-image Generationcited by 5
- 2025KB-DMGen: Knowledge-Based Global Guidance and Dynamic Pose Masking for Human Image Generationcited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2025Stitch: Training-Free Position Control in Multimodal Diffusion Transformers
- 2026MMFace-DiT: A Dual-Stream Diffusion Transformer for High-Fidelity Multimodal Face Generation
- 2026PoseAdapter: Dual-Stream 2.5D Controllable Image Generation for Complex Multi-Object Scenes
- 2024From Text to Pose to Image: Improving Diffusion Model Control and Quality
- 2024VividDreamer: Towards High-Fidelity and Efficient Text-to-3D Generation
- 2026Timo: Taming Multimodal Diffusion Transformer for Human Motion Generation
- 2024CPA: Camera-pose-awareness Diffusion Transformer for Video Generation
- 2025E-MMDiT: Revisiting Multimodal Diffusion Transformer Design for Fast Image Synthesis under Limited Resources