mlsanityData to 7 Oct 2026

Multilingual Conceptual Coverage in Text-to-Image Models

arXiv 2306.01735 ↗First posted 2 Jun 2023cs.CL cs.AI cs.CVcited by 9

Not on the map. Why?

Abstract (arXiv)

We propose "Conceptual Coverage Across Languages" (CoCo-CroLa), a technique for benchmarking the degree to which any generative text-to-image system provides multilingual parity to its training language in terms of tangible nouns. For each model we can assess "conceptual coverage" of a given target language relative to a source language by comparing the population of images generated for a series of tangible nouns in the source language to the population of images generated for each noun under translation in the target language. This technique allows us to estimate how well-suited a model is to a target language as well as identify model-specific weaknesses, spurious correlations, and biases without a-priori assumptions. We demonstrate how it can be used to benchmark T2I models in terms of multilinguality, and how despite its simplicity it is a good proxy for impressive generalization.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 36 of its 48 references that are in the map. First posted on arXiv 2 Jun 2023.

Builds on

36 of 48 references.

References · 36

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  6. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  7. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  8. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  9. 2022Generative Adversarial Networkscited by 5,603
  10. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  11. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  12. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
24 more
  1. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
  2. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  3. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,767
  4. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,544
  5. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,538
  6. 2018Datasheets for Datasetscited by 1,371
  7. 2018Model Cards for Model Reportingcited by 1,068
  8. 2020Beyond Accuracy: Behavioral Testing of NLP models with CheckListcited by 979
  9. 2017RACE: Large-scale ReAding Comprehension Dataset From Examinationscited by 851
  10. 2016Generative Adversarial Text to Image Synthesiscited by 735
  11. 2021CogView: Mastering Text-to-Image Generation via Transformerscited by 691
  12. 2019The Woman Worked as a Babysitter: On Biases in Language Generationcited by 540
  13. 2020Fine-Tuning Pretrained Language Models: Weight Initializations, Data Orders, and Early Stoppingcited by 377
  14. 2021AI and the Everything in the Whole Wide World Benchmarkcited by 340
  15. 2022Diffusion Models already have a Semantic Latent Spacecited by 324
  16. 2019Don't Take the Easy Way Out: Ensemble Based Methods for Avoiding Known Dataset Biasescited by 279
  17. 2022Easily Accessible Text-to-Image Generation Amplifies Demographic Stereotypes at Large Scalecited by 262
  18. 2020Data and its (dis)contents: A survey of dataset development and use in machine learning researchcited by 259
  19. 2019Effectively Unbiased FID and Inception Score and where to find themcited by 161
  20. 2015Generating Images from Captions with Attentioncited by 135
  21. 2022AltCLIP: Altering the Language Encoder in CLIP for Extended Language Capabilitiescited by 84
  22. 2022Underspecification in Scene Description-to-Depiction Taskscited by 30
  23. 2022Not All Errors are Equal: Learning Text Generation Metrics using Stratified Error Synthesiscited by 18
  24. 2021PECO: Examining Single Sentence Label Leakage in Natural Language Inference Datasets through Progressive Evaluation of Cluster Outlierscited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 4; 2025: 1; 2026: 1.

9 papers.

  1. 2024Who Evaluates the Evaluations? Objectively Scoring Text-to-Image Prompt Coherence Metrics with T2IScoreScore (TS2)cited by 17
  2. 2023Navigating Cultural Chasms: Exploring and Unlocking the Cultural POV of Text-To-Image Modelscited by 16
  3. 2024Multilingual Text-to-Image Generation Magnifies Gender Stereotypes and Prompt Engineering May Not Help Youcited by 15
  4. 2023ConceptBed: Evaluating Concept Learning Abilities of Text-to-Image Diffusion Modelscited by 10
  5. 2023On the Cultural Gap in Text-to-Image Generationcited by 10
  6. 2024Navigating Text-to-Image Generative Bias across Indic Languagescited by 5
  7. 2024M3Face: A Unified Multi-Modal Multilingual Framework for Human Face Generation and Editingcited by 3
  8. 2025CAIRe: Cultural Attribution of Images by Retrieval-Augmented Evaluationcited by 1
  9. 2026BAFIS: Dataset + Framework to assess occupational Bias and Human Preference in modern Text-to-image Modelscited by 0