mlsanityData to 7 Oct 2026

Cross-view Masked Diffusion Transformers for Person Image Synthesis

arXiv 2402.01516 ↗First posted 2 Feb 2024cs.CVcited by 12

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

We present X-MDPT (underlineCross-view underlineMasked underlineDiffusion underlinePrediction underlineTransformers), a novel diffusion model designed for pose-guided human image generation. X-MDPT distinguishes itself by employing masked diffusion transformers that operate on latent patches, a departure from the commonly-used Unet structures in existing works. The model comprises three key modules: 1) a denoising diffusion Transformer, 2) an aggregation network that consolidates conditions into a single vector for the diffusion process, and 3) a mask cross-prediction module that enhances representation learning with semantic information from the reference image. X-MDPT demonstrates scalability, improving FID, SSIM, and LPIPS with larger models. Despite its simple design, our model outperforms state-of-the-art approaches on the DeepFashion dataset while exhibiting efficiency in terms of training parameters, training time, and inference speed. Our compact 33MB model achieves an FID of 7.42, surpassing a prior Unet latent diffusion approach (FID 8.07) using only 11× fewer parameters. Our best model surpasses the pixel-based diffusion with frac23 of the parameters and achieves 5.43 × faster inference. The code is available at https://github.com/trungpx/xmdpt.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 42 of its 55 references that are in the map. First posted on arXiv 2 Feb 2024.

Builds on

42 of 55 references.

Closest ancestors · 7

Same direction, newest first

  1. 2023MDTv2: Masked Diffusion Transformer is a Strong Image Synthesizer25 Mar
  2. 2022Scalable Diffusion Models with Transformers19 Dec
  3. 2022Classifier-Free Diffusion Guidance26 Jul
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  5. 2021Cascaded Diffusion Models for High Fidelity Image Generation30 May
  6. 2020Denoising Diffusion Implicit Models6 Oct
  7. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 35

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  4. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  5. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  6. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  7. 2022MaskGIT: Masked Generative Image Transformercited by 1,082
  8. 2016Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,061
  9. 2023Muse: Text-To-Image Generation via Masked Generative Transformerscited by 640
  10. 2023AnyDoor: Zero-shot Object-level Image Customizationcited by 411
  11. 2023Uni-ControlNet: All-in-One Control to Text-to-Image Diffusion Modelscited by 358
  12. 2022MAGVIT: Masked Generative Video Transformercited by 341
23 more
  1. 2017Pose Guided Person Image Generationcited by 251
  2. 2023DreamPose: Fashion Image-to-Video Synthesis via Stable Diffusioncited by 167
  3. 2017Deformable GANs for Pose-based Human Image Generationcited by 166
  4. 2020Controllable Person Image Synthesis with Attribute-Decomposed GANcited by 150
  5. 2019Progressive Pose Attention Transfer for Person Image Generationcited by 138
  6. 2022Person Image Synthesis via Denoising Diffusion Modelcited by 104
  7. 2020Deep Image Spatial Transformation for Person Image Generationcited by 100
  8. 2023Siamese Masked Autoencoderscited by 75
  9. 2022Exploring Dual-task Correlation for Pose Guided Person Image Generationcited by 68
  10. 2022How Does SimSiam Avoid Collapse Without Negative Samples? A Unified Understanding with Self-supervised Contrastive Learningcited by 65
  11. 2021PISE: Person Image Synthesis and Editing with Decoupled GANcited by 62
  12. 2019Cascade RPN: Delving into High-Quality Region Proposal Network with Adaptive Convolutioncited by 57
  13. 2022Neural Texture Extraction and Distribution for Controllable Person Image Synthesiscited by 55
  14. 2022Dual Temperature Helps Contrastive Learning Without Many Negative Samples: Towards Understanding and Simplifying MoCocited by 52
  15. 2022Cross Attention Based Style Distribution for Controllable Person Image Synthesiscited by 45
  16. 2023CDPMSR: Conditional Diffusion Probabilistic Models for Single Image Super-Resolutioncited by 21
  17. 2023ACDMSR: Accelerated Conditional Diffusion Models for Single Image Super-Resolutioncited by 18
  18. 2022On the Pros and Cons of Momentum Encoder in Self-Supervised Visual Representation Learningcited by 10
  19. 2023Learning from Multi-Perception Features for Real-Word Image Super-resolutioncited by 9
  20. 2022LAD: A Hybrid Deep Learning System for Benign Paroxysmal Positional Vertigo Disorders Diagnosticcited by 8
  21. 2022Self-Supervised Visual Representation Learning via Residual Momentumcited by 7
  22. 2021Self-supervised Learning with Local Attention-Aware Featurecited by 5
  23. 2022Pose Guided Human Image Synthesis with Partially Decoupled GANcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 6; 2025: 4; 2026: 2.

12 papers.

  1. 2024Masked Image Modeling: A Surveycited by 28
  2. 2024Learning Flow Fields in Attention for Controllable Person Image Generationcited by 27
  3. 2025ITA-MDT: Image-Timestep-Adaptive Masked Diffusion Transformer Framework for Image-Based Virtual Try-Oncited by 10
  4. 2024MDSGen: Fast and Efficient Masked Diffusion Temporal-Aware Transformers for Open-Domain Sound Generationcited by 9
  5. 2024Adaptive Multi-Modal Control of Digital Human Hand Synthesis Using a Region-Aware Cycle Losscited by 2
  6. 2025E-MD3C: Taming Masked Diffusion Transformers for Efficient Zero-Shot Object Customizationcited by 2
  7. 2026A Hidden Semantic Bottleneck in Conditional Embeddings of Diffusion Transformerscited by 2
  8. 2024Bridging the Intent Gap: Knowledge-Enhanced Visual Generationcited by 1
  9. 2024Fusion Embedding for Pose-Guided Person Image Synthesis with Diffusion Modelcited by 0
  10. 2025Real-Time Person Image Synthesis Using a Flow Matching Modelcited by 0
  11. 2025Model-Guided Dual-Role Alignment for High-Fidelity Open-Domain Video-to-Audio Generationcited by 0
  12. 2026Astrolabe: Spherical-Map Guidance Across Diffusion Pipelines for Full-Body Capture from Unconstrained Imagescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Jointly Conditioned Diffusion Model for Multi-View Pose-Guided Person Image Synthesis
  2. 2026Nemotron-Labs-Diffusion-Image: Advancing Masked Discrete Diffusion for High-Resolution Image Synthesis
  3. 2022Exploring Vision Transformers as Diffusion Learners
  4. 2025Multi-focal Conditioned Latent Diffusion for Person Image Synthesis
  5. 2023Fast Training of Diffusion Models with Masked Transformers
  6. 2024EDT: An Efficient Diffusion Transformer Framework Inspired by Human-like Sketching
  7. 2026One-Shot Novel View and Pose Human Image Synthesis via 3D Prior Guided Diffusion Model
  8. 2025MC-VTON: Minimal Control Virtual Try-On Diffusion Transformer