# LumiGen: An LVLM-Enhanced Iterative Framework for Fine-Grained Text-to-Image Generation

arXiv: https://arxiv.org/abs/2508.04732 · HTML: https://mlsanity.com/paper/2508.04732/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models.
- Builds on 25 of its 32 references that are in the map.
- First posted on arXiv 5 Aug 2025.

## Placed in

- Diffusion models / [Image generation, text-to-image and editing](https://mlsanity.com/diffusion/image-generation-text-to-image-and-editing/)

## Builds on (most cited first, 25 of 25)

- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning](https://mlsanity.com/paper/2305.06500/) (2023)
- [Scaling Autoregressive Models for Content-Rich Text-to-Image Generation](https://mlsanity.com/paper/2206.10789/) (2022)
- [Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scaling](https://mlsanity.com/paper/2501.17811/) (2025)
- [OmniGen: Unified Image Generation](https://mlsanity.com/paper/2409.11340/) (2024)
- [LVLM-eHub: A Comprehensive Evaluation Benchmark for Large Vision-Language Models](https://mlsanity.com/paper/2306.09265/) (2023)
- [LAFITE: Towards Language-Free Training for Text-to-Image Generation](https://mlsanity.com/paper/2111.13792/) (2021)
- [Visual In-Context Learning for Large Vision-Language Models](https://mlsanity.com/paper/2402.11574/) (2024)
- [Diffusion Models in Low-Level Vision: A Survey](https://mlsanity.com/paper/2406.11138/) (2024)
- [Exploring the Frontier of Vision-Language Models: A Survey of Current Methodologies and Future Directions](https://mlsanity.com/paper/2404.07214/) (2024)
- [PromptMagician: Interactive Prompt Engineering for Text-to-Image Creation](https://mlsanity.com/paper/2307.09036/) (2023)
- [IdealGPT: Iteratively Decomposing Vision and Language Reasoning via Large Language Models](https://mlsanity.com/paper/2305.14985/) (2023)
- [Improving Medical Large Vision-Language Models with Abnormal-Aware Feedback](https://mlsanity.com/paper/2501.01377/) (2025)
- [MAM: Modular Multi-Agent Framework for Multi-Modal Medical Diagnosis via Role-Specialized Collaboration](https://mlsanity.com/paper/2506.19835/) (2025)
- [Text2Layer: Layered Image Generation using Latent Diffusion Model](https://mlsanity.com/paper/2307.09781/) (2023)
- [Divide, Evaluate, and Refine: Evaluating and Improving Text-to-Image Alignment with Iterative VQA Feedback](https://mlsanity.com/paper/2307.04749/) (2023)
- [Draw ALL Your Imagine: A Holistic Benchmark and Agent Framework for Complex Instruction-based Image Generation](https://mlsanity.com/paper/2505.24787/) (2025)
- [A Comprehensive Survey and Guide to Multimodal Large Language Models in Vision-Language Tasks](https://mlsanity.com/paper/2411.06284/) (2024)
- [Style-Aware Contrastive Learning for Multi-Style Image Captioning](https://mlsanity.com/paper/2301.11367/) (2023)
- [ComplexBench-Edit: Benchmarking Complex Instruction-Driven Image Editing via Compositional Dependencies](https://mlsanity.com/paper/2506.12830/) (2025)
- [VisLingInstruct: Elevating Zero-Shot Learning in Multi-Modal Language Models with Autonomous Instruction Optimization](https://mlsanity.com/paper/2402.07398/) (2024)
- [Divide-Then-Aggregate: An Efficient Tool Learning Method via Parallel Tool Invocation](https://mlsanity.com/paper/2501.12432/) (2025)
- [Text2Street: Controllable Text-to-image Generation for Street Views](https://mlsanity.com/paper/2402.04504/) (2024)
- [SDA: Simple Discrete Augmentation for Contrastive Sentence Representation Learning](https://mlsanity.com/paper/2210.03963/) (2022)
- [Visual Prompting with Iterative Refinement for Design Critique Generation](https://mlsanity.com/paper/2412.16829/) (2024)

## Built on by (most cited first, 1 of 1)

- [SoPE: Spherical Coordinate-Based Positional Embedding for Enhancing Spatial Perception of 3D LVLMs](https://mlsanity.com/paper/2602.22716/) (2026)

## Abstract (arXiv)

Text-to-Image (T2I) generation has made significant advancements with diffusion models, yet challenges persist in handling complex instructions, ensuring fine-grained content control, and maintaining deep semantic consistency. Existing T2I models often struggle with tasks like accurate text rendering, precise pose generation, or intricate compositional coherence. Concurrently, Vision-Language Models (LVLMs) have demonstrated powerful capabilities in cross-modal understanding and instruction following. We propose LumiGen, a novel LVLM-enhanced iterative framework designed to elevate T2I model performance, particularly in areas requiring fine-grained control, through a closed-loop, LVLM-driven feedback mechanism. LumiGen comprises an Intelligent Prompt Parsing & Augmentation (IPPA) module for proactive prompt enhancement and an Iterative Visual Feedback & Refinement (IVFR) module, which acts as a "visual critic" to iteratively correct and optimize generated images. Evaluated on the challenging LongBench-T2I Benchmark, LumiGen achieves a superior average score of 3.08, outperforming state-of-the-art baselines. Notably, our framework demonstrates significant improvements in critical dimensions such as text rendering and pose expression, validating the effectiveness of LVLM integration for more controllable and higher-quality image generation.