# Implicit Diversity in Image Summarization

arXiv: https://arxiv.org/abs/1901.10265 · HTML: https://mlsanity.com/paper/1901.10265/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Jan 2019.

## Built on by (most cited first, 18 of 18)

- [PRISM: A Rich Class of Parameterized Submodular Information Measures for Guided Subset Selection](https://mlsanity.com/paper/2103.00128/) (2021)
- [Survey of Social Bias in Vision-Language Models](https://mlsanity.com/paper/2309.14381/) (2023)
- [A Sociotechnical View of Algorithmic Fairness](https://mlsanity.com/paper/2110.09253/) (2021)
- [When Fair Ranking Meets Uncertain Inference](https://mlsanity.com/paper/2105.02091/) (2021)
- [Consensus and Subjectivity of Skin Tone Annotation for ML Fairness](https://mlsanity.com/paper/2305.09073/) (2023)
- [Mitigating Test-Time Bias for Fair Image Retrieval](https://mlsanity.com/paper/2305.19329/) (2023)
- [Fair Ranking with Noisy Protected Attributes](https://mlsanity.com/paper/2211.17067/) (2022)
- [Fairness and Bias Mitigation in Computer Vision: A Survey](https://mlsanity.com/paper/2408.02464/) (2024)
- [Skin Deep: Investigating Subjectivity in Skin Tone Annotations for Computer Vision Benchmark Datasets](https://mlsanity.com/paper/2305.09072/) (2023)
- [Generalized People Diversity: Learning a Human Perception-Aligned Diversity Representation for People Images](https://mlsanity.com/paper/2401.14322/) (2024)
- [Can we Debias Social Stereotypes in AI-Generated Images? Examining Text-to-Image Outputs and User Perceptions](https://mlsanity.com/paper/2505.20692/) (2025)
- [Subverting Fair Image Search with Generative Adversarial Perturbations](https://mlsanity.com/paper/2205.02414/) (2022)
- [Position: Certified Robustness Does Not (Yet) Imply Model Security](https://mlsanity.com/paper/2506.13024/) (2025)
- [Challenges and Solutions in AI for All](https://mlsanity.com/paper/2307.10600/) (2023)
- [Auditing for Diversity using Representative Examples](https://mlsanity.com/paper/2107.07393/) (2021)
- [Sampling Individually-Fair Rankings that are Always Group Fair](https://mlsanity.com/paper/2306.11964/) (2023)
- [Multi-Group Proportional Representation in Retrieval](https://mlsanity.com/paper/2407.08571/) (2024)
- [FaceSaliencyAug: Mitigating Geographic, Gender and Stereotypical Biases via Saliency-Based Data Augmentation](https://mlsanity.com/paper/2410.14070/) (2024)

## Abstract (arXiv)

Studies have shown that the people depicted in image search results tend to be of majority groups with respect to socially salient attributes. This skew goes beyond that which already exists in the world - e.g., Kay et al. showed that although 28% of CEOs in US are women, only 10% of the top 100 results for CEO in Google Image Search are women. Most existing approaches to correct for this kind of bias assume that the images of people include socially salient attribute labels. However, such labels are often unknown. Further, using automated techniques to infer these labels may often not be possible within acceptable accuracy ranges, and may not be desirable due to the additional biases this process could incur. We develop a novel approach that takes as input a visibly diverse control set of images and uses this set to select a set of images of people in response to a query. The goal is to have a resulting set that is more visibly diverse in a manner that emulates the diversity depicted in the control set. Importantly, this approach does not require images to be labelled at any point; effectively, it gives a way to implicitly diversify the set of images selected. We provide two variants of our approach: the first is a modification of the MMR algorithm to incorporate the diversity scores, and second is a more efficient variant that does not consider within-list redundancy. We evaluate these approaches empirically on two datasets 1) a new dataset containing top Google image results for 96 occupations, for which we evaluate gender and skin-tone diversity with respect to occupations and 2) the CelebA dataset for which we evaluate gender diversity with respect to facial features. Our approaches produce image sets that significantly improve the visible diversity of the results, compared to current Google search and other diverse image summarization algorithms, at a minimal cost to accuracy.