mlsanityData to 14 Oct 2026

Hierarchical Vision-Language Alignment for Text-to-Image Generation via Diffusion Models

arXiv 2501.00917 ↗First posted 1 Jan 2025cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → Image editing and quality assessment

also in 1 more
Abstract (arXiv)

Text-to-image generation has witnessed significant advancements with the integration of Large Vision-Language Models (LVLMs), yet challenges remain in aligning complex textual descriptions with high-quality, visually coherent images. This paper introduces the Vision-Language Aligned Diffusion (VLAD) model, a generative framework that addresses these challenges through a dual-stream strategy combining semantic alignment and hierarchical diffusion. VLAD utilizes a Contextual Composition Module (CCM) to decompose textual prompts into global and local representations, ensuring precise alignment with visual features. Furthermore, it incorporates a multi-stage diffusion process with hierarchical guidance to generate high-fidelity images. Experiments conducted on MARIO-Eval and INNOVATOR-Eval benchmarks demonstrate that VLAD significantly outperforms state-of-the-art methods in terms of image quality, semantic alignment, and text rendering accuracy. Human evaluations further validate the superior performance of VLAD, making it a promising approach for text-to-image generation in complex scenarios.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Image editing and quality assessment, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 23 of its 28 references that are in the map. First posted on arXiv 1 Jan 2025.

Builds on

23 of 28 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Towards Language-Driven Video Inpainting via Multimodal Large Language Models18 Jan

Other references · 22

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  3. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  4. 2023Muse: Text-To-Image Generation via Masked Generative Transformerscited by 648
  5. 2023VisionLLM: Large Language Model is also an Open-Ended Decoder for Vision-Centric Taskscited by 501
  6. 2024MoE-LLaVA: Mixture of Experts for Large Vision-Language Modelscited by 300
  7. 2023TextDiffuser: Diffusion Models as Text Painterscited by 229
  8. 2024InternLM-XComposer-2.5: A Versatile Large Vision Language Model Supporting Long-Contextual Input and Outputcited by 195
  9. 2024Visual In-Context Learning for Large Vision-Language Modelscited by 128
  10. 2024Training-Free Consistent Text-to-Image Generationcited by 125
  11. 2024Understanding Reinforcement Learning-Based Fine-Tuning of Diffusion Models: A Tutorial and Reviewcited by 98
  12. 2024A Survey on Benchmarks of Multimodal Large Language Modelscited by 82
10 more
  1. 2024Exploring the Frontier of Vision-Language Models: A Survey of Current Methodologies and Future Directionscited by 68
  2. 2024Rethinking Visual Dependency in Long-Context Reasoning for Large Vision-Language Modelscited by 42
  3. 2024Region-Aware Text-to-Image Generation via Hard Binding and Soft Refinementcited by 37
  4. 2024STAR: Scale-wise Text-conditioned AutoRegressive image generationcited by 28
  5. 2023Prompt Expansion for Adaptive Text-to-Image Generationcited by 24
  6. 2023Multimodal Event Transformer for Image-guided Story Ending Generationcited by 21
  7. 2023Improving Cross-modal Alignment for Text-Guided Image Inpaintingcited by 21
  8. 2023Style-Aware Contrastive Learning for Multi-Style Image Captioningcited by 21
  9. 2024Advancing Fine-Grained Visual Understanding with Multi-Scale Alignment in Multi-Modal Modelscited by 11
  10. 2024Generalist Multimodal AI: A Review of Architectures, Challenges and Opportunitiescited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023Improving Compositional Text-to-image Generation with Large Vision-Language Models
  2. 2024Bridging Different Language Models and Generative Vision Models for Text-to-Image Generation
  3. 2025HCMA: Hierarchical Cross-model Alignment for Grounded Text-to-Image Generation
  4. 2023Divide, Evaluate, and Refine: Evaluating and Improving Text-to-Image Alignment with Iterative VQA Feedback
  5. 2025M3T2IBench: A Large-Scale Multi-Category, Multi-Instance, Multi-Relation Text-to-Image Benchmark
  6. 2023Paragraph-to-Image Generation with Information-Enriched Diffusion Model
  7. 2024An Intermediate Fusion ViT Enables Efficient Text-Image Alignment in Diffusion Models
  8. 2024Noise Diffusion for Enhancing Semantic Faithfulness in Text-to-Image Synthesis