# The Describe-Then-Generate Bottleneck: How VLM Descriptions Alter Image Generation Outcomes

arXiv: https://arxiv.org/abs/2509.18179 · HTML: https://mlsanity.com/paper/2509.18179/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 35 references that are in the map.
- First posted on arXiv 18 Sep 2025.

## Builds on (most cited first, 25 of 25)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Concept Bottleneck Models](https://mlsanity.com/paper/2007.04612/) (2020)
- [DeCo: Decoupling Token Compression from Semantic Abstraction in Multimodal Large Language Models](https://mlsanity.com/paper/2405.20985/) (2024)
- [Cross-modal Information Flow in Multimodal Large Language Models](https://mlsanity.com/paper/2411.18620/) (2024)
- [Extending Context Window of Large Language Models via Semantic Compression](https://mlsanity.com/paper/2312.09571/) (2023)
- [Geometric Multimodal Contrastive Representation Learning](https://mlsanity.com/paper/2202.03390/) (2022)
- [Looking Beyond Text: Reducing Language bias in Large Vision-Language Models via Multimodal Dual-Attention and Soft-Image Guidance](https://mlsanity.com/paper/2411.14279/) (2024)
- [Identifying and Mitigating Position Bias of Multi-image Vision-Language Models](https://mlsanity.com/paper/2503.13792/) (2025)
- [Lost in Embeddings: Information Loss in Vision-Language Models](https://mlsanity.com/paper/2509.11986/) (2025)
- [Principled Multimodal Representation Learning](https://mlsanity.com/paper/2507.17343/) (2025)
- [See It from My Perspective: How Language Affects Cultural Bias in Image Understanding](https://mlsanity.com/paper/2406.11665/) (2024)
- [DCBM: Data-Efficient Visual Concept Bottleneck Models](https://mlsanity.com/paper/2412.11576/) (2024)
- [Learning Optimal Multimodal Information Bottleneck Representations](https://mlsanity.com/paper/2505.19996/) (2025)
- [Dynamic Multimodal Information Bottleneck for Multimodality Classification](https://mlsanity.com/paper/2311.01066/) (2023)
- [Improving Image Autoencoder Embeddings with Perceptual Loss](https://mlsanity.com/paper/2001.03444/) (2020)
- [Aligning Multimodal Representations through an Information Bottleneck](https://mlsanity.com/paper/2506.04870/) (2025)
- [Free-VSC: Free Semantics from Visual Foundation Models for Unsupervised Video Semantic Compression](https://mlsanity.com/paper/2409.11718/) (2024)
- [VisuRiddles: Fine-grained Perception is a Primary Bottleneck for Multimodal Large Language Models in Abstract Visual Reasoning](https://mlsanity.com/paper/2506.02537/) (2025)
- [Multimodal Conditional Information Bottleneck for Generalizable AI-Generated Image Detection](https://mlsanity.com/paper/2505.15217/) (2025)
- [Uncovering Cultural Representation Disparities in Vision-Language Models](https://mlsanity.com/paper/2505.14729/) (2025)
- [The Right Losses for the Right Gains: Improving the Semantic Consistency of Deep Text-to-Image Generation with Distribution-Sensitive Losses](https://mlsanity.com/paper/2312.10854/) (2023)

## Built on by (most cited first, 1 of 1)

- [Do Foundation Models Know Geometry? Probing Frozen Features for Continuous Physical Measurement](https://mlsanity.com/paper/2603.06459/) (2026)

## Abstract (arXiv)

With the increasing integration of multimodal AI systems in creative workflows, understanding information loss in vision-language-vision pipelines has become important for evaluating system limitations. However, the degradation that occurs when visual content passes through textual intermediation remains poorly quantified. In this work, we provide empirical analysis of the describe-then-generate bottleneck, where natural language serves as an intermediate representation for visual information. We generated 150 image pairs through the describe-then-generate pipeline and applied existing metrics (LPIPS, SSIM, and color distance) to measure information preservation across perceptual, structural, and chromatic dimensions. Our evaluation reveals that 99.3% of samples exhibit substantial perceptual degradation and 91.5% demonstrate significant structural information loss, providing empirical evidence that the describe-then-generate bottleneck represents a measurable and consistent limitation in contemporary multimodal systems.