mlsanityData to 7 Oct 2026

TrioPose: Native Triple-Stream Diffusion Transformers for Pose-Guided Text-to-Image Generation

arXiv 2606.07053 ↗First posted 5 Jun 2026cs.CV cs.LGcited by 1

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

Pose-guided text-to-image generation often suffers from limb distortions and feature crosstalk in complex multi-person scenarios. While existing UNet-based adapters struggle with long-range spatial dependencies, emerging Multimodal Diffusion Transformers (MM-DiTs) offer superior global modeling. However, naive signal concatenation in MM-DiTs severely disrupts pre-trained latent distributions. To address this, we propose TrioPose, a native pose-driven framework built upon the SD3.5M architecture. Specifically, we introduce a Triple-Stream Pose-Aware DiT (TSPA-DiT) that treats pose as an independent modality. It employs layer-wise activation and zero-initialized dual-residual injection to smoothly enforce geometric constraints while preserving pre-trained latent stability. To resolve severe multi-instance occlusions, we design a Learnable Relational Bias Mask that categorizes topological connectivity into fine-grained physical states, mapping them into continuous attention soft constraints to effectively decouple inter-instance interference. Furthermore, a Pose-Guided Spatial Loss Weighting strategy modulates the native diffusion objective using heatmap-derived error maps, focusing anatomical supervision strictly on distortion-prone regions. Extensive experiments demonstrate that TrioPose achieves state-of-the-art performance across challenging benchmarks, including Human-Art, CrowdPose, and OCHuman. Notably, it attains an AP of 64.33 on Human-Art, representing a 30% improvement over prior arts, while setting new standards for visual fidelity and text-image semantic alignment in complex multi-human generation.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 33 of its 42 references that are in the map. First posted on arXiv 5 Jun 2026.

Builds on

33 of 42 references.

Closest ancestors · 7

Same direction, newest first

  1. 2025PixelPonder: Dynamic Patch Adaptation for Enhanced Multi-Conditional Text-to-Image Generation9 Mar
  2. 2023Uni-ControlNet: All-in-One Control to Text-to-Image Diffusion Models25 May
  3. 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models16 Feb
  4. 2023Adding Conditional Control to Text-to-Image Diffusion Models10 Feb
  5. 2023GLIGEN: Open-Set Grounded Text-to-Image Generation17 Jan
  6. 2022Synthesizing Coherent Story with Auto-Regressive Latent Diffusion Models20 Nov
  7. 2022Hierarchical Text-Conditional Image Generation with CLIP Latents13 Apr

Other references · 26

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  5. 2022Scalable Diffusion Models with Transformerscited by 5,931
  6. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
  7. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
  8. 2025Qwen3-VL Technical Reportcited by 2,534
  9. 2018Demystifying MMD GANscited by 1,268
  10. 2024Champ: Controllable and Consistent Human Image Animation with 3D Parametric Guidancecited by 271
  11. 2017Pose Guided Person Image Generationcited by 251
  12. 2018CrowdPose: Efficient Crowded Scenes Pose Estimation and A New Benchmarkcited by 214
14 more
  1. 2020Controllable Person Image Synthesis with Attribute-Decomposed GANcited by 150
  2. 2019Progressive Pose Attention Transfer for Person Image Generationcited by 138
  3. 2018Pose2Seg: Detection Free Human Instance Segmentationcited by 107
  4. 2023HumanSD: A Native Skeleton-Guided Diffusion Model for Human Image Generationcited by 104
  5. 2023Human-Art: A Versatile Human-Centric Dataset Bridging Natural and Artificial Scenescited by 68
  6. 2022ViTPose++: Vision Transformer for Generic Body Pose Estimationcited by 65
  7. 2021PISE: Person Image Synthesis and Editing with Decoupled GANcited by 62
  8. 2017Be Your Own Prada: Fashion Synthesis with Structural Coherencecited by 49
  9. 2021Learning Semantic Person Image Generation by Region-Adaptive Normalizationcited by 44
  10. 2024GRPose: Learning Graph Relations for Human Image Generation with Pose Priorscited by 14
  11. 2022KPE: Keypoint Pose Encoding for Transformer-based Image Generationcited by 12
  12. 2024Stable-Pose: Leveraging Transformers for Pose-Guided Text-to-Image Generationcited by 12
  13. 2025Rethink Sparse Signals for Pose-guided Text-to-image Generationcited by 5
  14. 2025KB-DMGen: Knowledge-Based Global Guidance and Dynamic Pose Masking for Human Image Generationcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2025Stitch: Training-Free Position Control in Multimodal Diffusion Transformers
  2. 2026MMFace-DiT: A Dual-Stream Diffusion Transformer for High-Fidelity Multimodal Face Generation
  3. 2026PoseAdapter: Dual-Stream 2.5D Controllable Image Generation for Complex Multi-Object Scenes
  4. 2024From Text to Pose to Image: Improving Diffusion Model Control and Quality
  5. 2024VividDreamer: Towards High-Fidelity and Efficient Text-to-3D Generation
  6. 2026Timo: Taming Multimodal Diffusion Transformer for Human Motion Generation
  7. 2024CPA: Camera-pose-awareness Diffusion Transformer for Video Generation
  8. 2025E-MMDiT: Revisiting Multimodal Diffusion Transformer Design for Fast Image Synthesis under Limited Resources