# Semantic Browsing: Controllable Diversity for Image Generation

arXiv: https://arxiv.org/abs/2606.23679 · HTML: https://mlsanity.com/paper/2606.23679/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 33 of its 67 references that are in the map.
- First posted on arXiv 22 Jun 2026.

## Builds on (most cited first, 30 of 33)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [Improving Factuality and Reasoning in Language Models through Multiagent Debate](https://mlsanity.com/paper/2305.14325/) (2023)
- [Evaluating Text-to-Visual Generation with Image-to-Text Generation](https://mlsanity.com/paper/2404.01291/) (2024)
- [The Vendi Score: A Diversity Evaluation Metric for Machine Learning](https://mlsanity.com/paper/2210.02410/) (2022)
- [Guiding a Diffusion Model with a Bad Version of Itself](https://mlsanity.com/paper/2406.02507/) (2024)
- [CADS: Unleashing the Diversity of Diffusion Models through Condition-Annealed Sampling](https://mlsanity.com/paper/2310.17347/) (2023)
- [Prometheus-Vision: Vision-Language Model as a Judge for Fine-Grained Evaluation](https://mlsanity.com/paper/2401.06591/) (2024)
- [Concept Decomposition for Visual Exploration and Inspiration](https://mlsanity.com/paper/2305.18203/) (2023)
- [ConceptLab: Creative Concept Generation using VLM-Guided Diffusion Prior Constraints](https://mlsanity.com/paper/2308.02669/) (2023)
- [Proactive Agents for Multi-Turn Text-to-Image Generation Under Uncertainty](https://mlsanity.com/paper/2412.06771/) (2024)
- [Distilling Diversity and Control in Diffusion Models](https://mlsanity.com/paper/2503.10637/) (2025)
- [Maestro: Self-Improving Text-to-Image Generation via Agent Orchestration](https://mlsanity.com/paper/2509.10704/) (2025)
- [PromptSculptor: Multi-Agent Based Text-to-Image Prompt Optimization](https://mlsanity.com/paper/2509.12446/) (2025)
- [Minority-Focused Text-to-Image Generation via Prompt Optimization](https://mlsanity.com/paper/2410.07838/) (2024)
- [Stage-wise Dynamics of Classifier-Free Guidance in Diffusion Models](https://mlsanity.com/paper/2509.22007/) (2025)
- [Generating an Image From 1,000 Words: Enhancing Text-to-Image With Structured Captions](https://mlsanity.com/paper/2511.06876/) (2025)
- [From Posterior Sampling to Meaningful Diversity in Image Restoration](https://mlsanity.com/paper/2310.16047/) (2023)
- [Scaling Group Inference for Diverse and High-Quality Generation](https://mlsanity.com/paper/2508.15773/) (2025)
- [Twin Co-Adaptive Dialogue for Progressive Image Generation](https://mlsanity.com/paper/2504.14868/) (2025)
- [Navigating with Annealing Guidance Scale in Diffusion Space](https://mlsanity.com/paper/2506.24108/) (2025)
- [Be Decisive: Noise-Induced Layouts for Multi-Subject Generation](https://mlsanity.com/paper/2505.21488/) (2025)
- [Hierarchical Uncertainty Exploration via Feedforward Posterior Trees](https://mlsanity.com/paper/2405.15719/) (2024)
- [On-the-fly Repulsion in the Contextual Space for Rich Diversity in Diffusion Transformers](https://mlsanity.com/paper/2603.28762/) (2026)

## Abstract (arXiv)

Modern text-to-image models excel in visual fidelity and prompt adherence. However, this strict adherence comes at the cost of diversity: generated samples tend to collapse into a single visual interpretation. Existing methods to improve diversity produce outputs driven by incidental variations rather than meaningful design choices. This motivates a new variant of the diversity task where structure is enforced on the generated samples. We introduce a method for controlled diversity that enables Semantic Browsing, where users can navigate structured image galleries and experience creative exploration through a systematic traversal of meaningful, interpretable axes of variation. Achieving this level of semantic control requires a deep understanding of the scene. We exploit the fact that recent text-to-image models are trained on elaborated captions, effectively decoupling semantic decision-making from pixel generation. This enables a paradigm shift: instead of relying on stochastic variation within the text-to-image model, we induce diversity directly at the text level. By leveraging rich textual representations, we allow a Vision Language Model (VLM) to operate on the full scene context. To overcome the generic outputs typical of standard VLMs, we employ an agentic workflow that explicitly enforces structured variation attuned to the original prompt. We demonstrate that our method produces diverse and navigable design spaces where every variation corresponds to a specific, user-understandable semantic decision.