# Decomposed evaluations of geographic disparities in text-to-image models

arXiv: https://arxiv.org/abs/2406.11988 · HTML: https://mlsanity.com/paper/2406.11988/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 14 references that are in the map.
- First posted on arXiv 17 Jun 2024.

## Builds on (most cited first, 12 of 12)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023)
- [Improved Precision and Recall Metric for Assessing Generative Models](https://mlsanity.com/paper/1904.06991/) (2019)
- [Reliable Fidelity and Diversity Metrics for Generative Models](https://mlsanity.com/paper/2002.09797/) (2020)
- [StableRep: Synthetic Images from Text-to-Image Models Make Strong Visual Representation Learners](https://mlsanity.com/paper/2306.00984/) (2023)
- [How well can Text-to-Image Generative Models understand Ethical Natural Language Interventions?](https://mlsanity.com/paper/2210.15230/) (2022)
- [Inspecting the Geographical Representativeness of Images from Text-to-Image Models](https://mlsanity.com/paper/2305.11080/) (2023)
- [Towards Geographic Inclusion in the Evaluation of Text-to-Image Models](https://mlsanity.com/paper/2405.04457/) (2024)
- [Controlled Training Data Generation with Diffusion Models](https://mlsanity.com/paper/2403.15309/) (2024)

## Built on by (most cited first, 4 of 4)

- [EvalGIM: A Library for Evaluating Generative Image Models](https://mlsanity.com/paper/2412.10604/) (2024)
- [CAIRe: Cultural Attribution of Images by Retrieval-Augmented Evaluation](https://mlsanity.com/paper/2506.09109/) (2025)
- [What's in Common? Multimodal Models Hallucinate When Reasoning Across Scenes](https://mlsanity.com/paper/2511.03768/) (2025)
- [GeoFidelity-Bench: Evaluating Block-Conditioned Geographic Fidelity in Text-to-Image Street-View Generation](https://mlsanity.com/paper/2606.23669/) (2026)

## Abstract (arXiv)

Recent work has identified substantial disparities in generated images of different geographic regions, including stereotypical depictions of everyday objects like houses and cars. However, existing measures for these disparities have been limited to either human evaluations, which are time-consuming and costly, or automatic metrics evaluating full images, which are unable to attribute these disparities to specific parts of the generated images. In this work, we introduce a new set of metrics, Decomposed Indicators of Disparities in Image Generation (Decomposed-DIG), that allows us to separately measure geographic disparities in the depiction of objects and backgrounds in generated images. Using Decomposed-DIG, we audit a widely used latent diffusion model and find that generated images depict objects with better realism than backgrounds and that backgrounds in generated images tend to contain larger regional disparities than objects. We use Decomposed-DIG to pinpoint specific examples of disparities, such as stereotypical background generation in Africa, struggling to generate modern vehicles in Africa, and unrealistically placing some objects in outdoor settings. Informed by our metric, we use a new prompting structure that enables a 52% worst-region improvement and a 20% average improvement in generated background diversity.