# KB-DMGen: Knowledge-Based Global Guidance and Dynamic Pose Masking for Human Image Generation

arXiv: https://arxiv.org/abs/2507.20083 · HTML: https://mlsanity.com/paper/2507.20083/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Video generation, one of 14 directions in Diffusion models.
- Builds on 39 of its 51 references that are in the map.
- First posted on arXiv 26 Jul 2025.

## Placed in

- Diffusion models / [Video generation](https://mlsanity.com/diffusion/video-generation/)

## Builds on (most cited first, 30 of 39)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020)
- [FiLM: Visual Reasoning with a General Conditioning Layer](https://mlsanity.com/paper/1709.07871/) (2017)
- [Generating Diverse High-Fidelity Images with VQ-VAE-2](https://mlsanity.com/paper/1906.00446/) (2019)
- [T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.08453/) (2023)
- [Reproducible scaling laws for contrastive language-image learning](https://mlsanity.com/paper/2212.07143/) (2022)
- [GLIGEN: Open-Set Grounded Text-to-Image Generation](https://mlsanity.com/paper/2301.07093/) (2023)
- [Autoencoding beyond pixels using a learned similarity metric](https://mlsanity.com/paper/1512.09300/) (2015)
- [Uni-ControlNet: All-in-One Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2305.16322/) (2023)
- [Composer: Creative and Controllable Image Synthesis with Composable Conditions](https://mlsanity.com/paper/2302.09778/) (2023)
- [Pose Guided Person Image Generation](https://mlsanity.com/paper/1705.09368/) (2017)
- [Controlling Text-to-Image Diffusion by Orthogonal Finetuning](https://mlsanity.com/paper/2306.07280/) (2023)
- [ControlNet++: Improving Conditional Controls with Efficient Consistency Feedback](https://mlsanity.com/paper/2404.07987/) (2024)
- [Controllable Person Image Synthesis with Attribute-Decomposed GAN](https://mlsanity.com/paper/2003.12267/) (2020)
- [HumanSD: A Native Skeleton-Guided Diffusion Model for Human Image Generation](https://mlsanity.com/paper/2304.04269/) (2023)
- [Person Image Synthesis via Denoising Diffusion Model](https://mlsanity.com/paper/2211.12500/) (2022)
- [Advancing Pose-Guided Image Synthesis with Progressive Conditional Diffusion Models](https://mlsanity.com/paper/2310.06313/) (2023)
- [XingGAN for Person Image Generation](https://mlsanity.com/paper/2007.09278/) (2020)
- [Synthesizing Coherent Story with Auto-Regressive Latent Diffusion Models](https://mlsanity.com/paper/2211.10950/) (2022)

## Built on by (most cited first, 1 of 1)

- [TrioPose: Native Triple-Stream Diffusion Transformers for Pose-Guided Text-to-Image Generation](https://mlsanity.com/paper/2606.07053/) (2026)

## Abstract (arXiv)

Recent methods using diffusion models have made significant progress in Human Image Generation (HIG) with various control signals such as pose priors. In HIG, both accurate human poses and coherent visual quality are crucial for image generation. However, most existing methods mainly focus on pose accuracy while neglecting overall image quality, often improving pose alignment at the cost of image quality. To address this, we propose Knowledge-Based Global Guidance and Dynamic pose Masking for human image Generation (KB-DMGen). The Knowledge Base (KB), implemented as a visual codebook, provides coarse, global guidance based on input text-related visual features, improving pose accuracy while maintaining image quality, while the Dynamic pose Mask (DM) offers fine-grained local control to enhance precise pose accuracy. By injecting KB and DM at different stages of the diffusion process, our framework enhances pose accuracy through both global and local control without compromising image quality. Experiments demonstrate the effectiveness of KB-DMGen, achieving new state-of-the-art results in terms of AP and CAP on the HumanArt dataset. The project page and code are available at https://lushbng.github.io/KBDMGen.