# Multimodal Benchmarking and Recommendation of Text-to-Image Generation Models

arXiv: https://arxiv.org/abs/2505.04650 · HTML: https://mlsanity.com/paper/2505.04650/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 13 references that are in the map.
- First posted on arXiv 6 May 2025.

## Builds on (most cited first, 12 of 12)

- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2308.06721/) (2023)
- [PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesis](https://mlsanity.com/paper/2310.00426/) (2023)
- [Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inference](https://mlsanity.com/paper/2310.04378/) (2023)
- [Lumina-Image 2.0: A Unified and Efficient Image Generative Framework](https://mlsanity.com/paper/2503.21758/) (2025)
- [SANA-Sprint: One-Step Diffusion with Continuous-Time Consistency Distillation](https://mlsanity.com/paper/2503.09641/) (2025)
- [CogView3: Finer and Faster Text-to-Image Generation via Relay Diffusion](https://mlsanity.com/paper/2403.05121/) (2024)
- [Text2Human: Text-Driven Controllable Human Image Generation](https://mlsanity.com/paper/2205.15996/) (2022)
- [Group Diffusion Transformers are Unsupervised Multitask Learners](https://mlsanity.com/paper/2410.15027/) (2024)
- [KOALA: Empirical Lessons Toward Memory-Efficient and Fast Diffusion Models for Text-to-Image Synthesis](https://mlsanity.com/paper/2312.04005/) (2023)

## Built on by (most cited first, 3 of 3)

- [A Comprehensive Dataset for Human vs. AI Generated Image Detection](https://mlsanity.com/paper/2601.00553/) (2026)
- [HuggingR4: A Progressive Reasoning Framework for Discovering Optimal Model Companions](https://mlsanity.com/paper/2511.18715/) (2025)
- [ECLIPTICA -- A Framework for Switchable LLM Alignment via CITA - Contrastive Instruction-Tuned Alignment](https://mlsanity.com/paper/2601.06157/) (2026)

## Abstract (arXiv)

This work presents an open-source unified benchmarking and evaluation framework for text-to-image generation models, with a particular focus on the impact of metadata augmented prompts. Leveraging the DeepFashion-MultiModal dataset, we assess generated outputs through a comprehensive set of quantitative metrics, including Weighted Score, CLIP (Contrastive Language Image Pre-training)-based similarity, LPIPS (Learned Perceptual Image Patch Similarity), FID (Frechet Inception Distance), and retrieval-based measures, as well as qualitative analysis. Our results demonstrate that structured metadata enrichments greatly enhance visual realism, semantic fidelity, and model robustness across diverse text-to-image architectures. While not a traditional recommender system, our framework enables task-specific recommendations for model selection and prompt design based on evaluation metrics.