mlsanityData to 13 Oct 2026

Is What You Ask For What You Get? Investigating Concept Associations in Text-to-Image Models

arXiv 2410.04634 ↗First posted 6 Oct 2024cs.CVcited by 2

Not on the map. Why?

Abstract (arXiv)

Text-to-image (T2I) models are increasingly used in impactful real-life applications. As such, there is a growing need to audit these models to ensure that they generate desirable, task-appropriate images. However, systematically inspecting the associations between prompts and generated content in a human-understandable way remains challenging. To address this, we propose Concept2Concept, a framework where we characterize conditional distributions of vision language models using interpretable concepts and metrics that can be defined in terms of these concepts. This characterization allows us to use our framework to audit models and prompt-datasets. To demonstrate, we investigate several case studies of conditional distributions of prompts, such as user-defined distributions or empirical, real-world distributions. Lastly, we implement Concept2Concept as an open-source interactive visualization tool to facilitate use by non-technical end-users. A demo is available at https://tinyurl.com/Concept2ConceptDemo.

Key facts. Cited by 2 papers in the map (data through 13 Oct 2026). Builds on 33 of its 38 references that are in the map. First posted on arXiv 6 Oct 2024.

Builds on

33 of 38 references.

References · 33

  1. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,476
  2. 2020End-to-End Object Detection with Transformerscited by 7,396
  3. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,314
  4. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,130
  5. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,299
  6. 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,316
  7. 2023Kosmos-2: Grounding Multimodal Large Language Models to the Worldcited by 1,082
  8. 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,028
  9. 2020Concept Bottleneck Modelscited by 1,027
  10. 2023Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generationcited by 961
  11. 2023Extracting Training Data from Diffusion Modelscited by 729
  12. 2023Florence-2: Advancing a Unified Representation for a Variety of Vision Taskscited by 406
21 more
  1. 2023Scaling Open-Vocabulary Object Detectioncited by 337
  2. 2023Synthetic Data from Diffusion Models Improves ImageNet Classificationcited by 327
  3. 2022Is synthetic data from generative models ready for image recognition?cited by 320
  4. 2022Easily Accessible Text-to-Image Generation Amplifies Demographic Stereotypes at Large Scalecited by 262
  5. 2022DALL-Eval: Probing the Reasoning Skills and Social Biases of Text-to-Image Generation Modelscited by 230
  6. 2023Stable Bias: Analyzing Societal Representations in Diffusion Modelscited by 226
  7. 2020Concept Whitening for Interpretable Image Recognitioncited by 225
  8. 2022Fake it till you make it: Learning transferable representations from synthetic ImageNet clonescited by 172
  9. 2023Rich Human Feedback for Text-to-Image Generationcited by 141
  10. 2024Lumina-T2X: Transforming Text into Any Modality, Resolution, and Duration via Flow-based Large Diffusion Transformerscited by 137
  11. 2023Leaving Reality to Imagination: Robust Classification via Generated Datasetscited by 91
  12. 2024OpenBias: Open-set Bias Detection in Text-to-Image Generative Modelscited by 61
  13. 2024Examining Gender and Racial Bias in Large Vision-Language Models Using a Novel Dataset of Parallel Imagescited by 55
  14. 2023Into the LAIONs Den: Investigating Hate in Multimodal Datasetscited by 54
  15. 2023SocialCounterfactuals: Probing and Mitigating Intersectional Social Biases in Vision-Language Models with Counterfactual Examplescited by 53
  16. 2023TIBET: Identifying and Evaluating Biases in Text-to-Image Generative Modelscited by 51
  17. 2023Balancing the Picture: Debiasing Vision-Language Datasets with Synthetic Contrast Setscited by 27
  18. 2023Stereotypes and Smut: The (Mis)representation of Non-cisgender Identities by Text-to-Image Modelscited by 27
  19. 2023Quantifying Bias in Text-to-Image Generative Modelscited by 16
  20. 2024They're All Doctors: Synthesizing Diverse Counterfactuals to Mitigate Associative Biascited by 3
  21. 2024CUPID: Contextual Understanding of Prompt-conditioned Image Distributionscited by 2

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Bias at the End of the Scorecited by 1
  2. 2026Position: Preventing AI-Generated CSAM Necessitates New Approaches to AI Safetycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023ConceptBed: Evaluating Concept Learning Abilities of Text-to-Image Diffusion Models
  2. 2025FineGRAIN: Evaluating Failure Modes of Text-to-Image Models with Vision Language Model Judges
  3. 2025ConceptMix++: Leveling the Playing Field in Text-to-Image Benchmarking via Iterative Prompt Optimization
  4. 2024Who Evaluates the Evaluations? Objectively Scoring Text-to-Image Prompt Coherence Metrics with T2IScoreScore (TS2)
  5. 2025Draw ALL Your Imagine: A Holistic Benchmark and Agent Framework for Complex Instruction-based Image Generation
  6. 2025Easier Painting Than Thinking: Can Text-to-Image Models Set the Stage, but Not Direct the Play?
  7. 2024ConceptMix: A Compositional Image Generation Benchmark with Controllable Difficulty
  8. 2025Why Settle for One? Text-to-ImageSet Generation and Evaluation