# IndicFairFace: Balanced Indian Face Dataset for Auditing and Mitigating Geographical Bias in Vision-Language Models

arXiv: https://arxiv.org/abs/2602.12659 · HTML: https://mlsanity.com/paper/2602.12659/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Social bias and fairness in VLMs, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 34 of its 52 references that are in the map.
- First posted on arXiv 13 Feb 2026.

## Placed in

- Multimodal LLMs / vision-language models / [Social bias and fairness in VLMs](https://mlsanity.com/mllm/social-bias-and-fairness-in-vlms/)

## Builds on (most cited first, 30 of 34)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Sigmoid Loss for Language Image Pre-Training](https://mlsanity.com/paper/2303.15343/) (2023)
- [Reproducible scaling laws for contrastive language-image learning](https://mlsanity.com/paper/2212.07143/) (2022)
- [AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty](https://mlsanity.com/paper/1912.02781/) (2019)
- [Vision-Language Models for Vision Tasks: A Survey](https://mlsanity.com/paper/2304.00685/) (2023)
- [Null It Out: Guarding Protected Attributes by Iterative Nullspace Projection](https://mlsanity.com/paper/2004.07667/) (2020)
- [Age Progression/Regression by Conditional Adversarial Autoencoder](https://mlsanity.com/paper/1702.08423/) (2017)
- [Towards Understanding and Mitigating Social Biases in Language Models](https://mlsanity.com/paper/2106.13219/) (2021)
- [Demystifying CLIP Data](https://mlsanity.com/paper/2309.16671/) (2023)
- [Gender Bias in Contextualized Word Embeddings](https://mlsanity.com/paper/1904.03310/) (2019)
- [No Classification without Representation: Assessing Geodiversity Issues in Open Data Sets for the Developing World](https://mlsanity.com/paper/1711.08536/) (2017)
- [On Measuring and Mitigating Biased Inferences of Word Embeddings](https://mlsanity.com/paper/1908.09369/) (2019)
- [Benchmarking Vision Language Models for Cultural Understanding](https://mlsanity.com/paper/2407.10920/) (2024)
- [MobileCLIP: Fast Image-Text Models through Multi-Modal Reinforced Training](https://mlsanity.com/paper/2311.17049/) (2023)
- [Diversity in Faces](https://mlsanity.com/paper/1901.10436/) (2019)
- [All Languages Matter: Evaluating LMMs on Culturally Diverse 100 Languages](https://mlsanity.com/paper/2411.16508/) (2024)
- [Inspecting the Geographical Representativeness of Images from Text-to-Image Models](https://mlsanity.com/paper/2305.11080/) (2023)
- [Spherical Linear Interpolation and Text-Anchoring for Zero-shot Composed Image Retrieval](https://mlsanity.com/paper/2405.00571/) (2024)
- [Heterogeneous Face Attribute Estimation: A Deep Multi-Task Learning Approach](https://mlsanity.com/paper/1706.00906/) (2017)
- [IndiBias: A Benchmark Dataset to Measure Social Biases in Language Models for Indian Context](https://mlsanity.com/paper/2403.20147/) (2024)
- [Identifying Implicit Social Biases in Vision-Language Models](https://mlsanity.com/paper/2411.00997/) (2024)
- [Building Socio-culturally Inclusive Stereotype Resources with Community Engagement](https://mlsanity.com/paper/2307.10514/) (2023)
- [MultiModal Bias: Introducing a Framework for Stereotypical Bias Assessment beyond Gender and Race in Vision Language Models](https://mlsanity.com/paper/2303.12734/) (2023)
- [Indian-BhED: A Dataset for Measuring India-Centric Biases in Large Language Models](https://mlsanity.com/paper/2309.08573/) (2023)
- [OpenVision: A Fully-Open, Cost-Effective Family of Advanced Vision Encoders for Multimodal Learning](https://mlsanity.com/paper/2505.04601/) (2025)
- [Chitrarth: Bridging Vision and Language for a Billion People](https://mlsanity.com/paper/2502.15392/) (2025)
- [FairI Tales: Evaluation of Fairness in Indian Contexts with a Focus on Bias and Stereotypes](https://mlsanity.com/paper/2506.23111/) (2025)
- [Culture-TRIP: Culturally-Aware Text-to-Image Generation with Iterative Prompt Refinement](https://mlsanity.com/paper/2502.16902/) (2025)

## Abstract (arXiv)

Vision-Language Models (VLMs) are known to inherit and amplify societal biases from their web-scale training data with Indian being particularly misrepresented. Existing fairness-aware datasets have significantly improved demographic balance across global race and gender groups, yet they continue to treat Indian as a single monolithic category. The oversimplification ignores the vast intra-national diversity across 28 states and 8 Union Territories of India and leads to representational and geographical bias. To address the limitation, we present IndicFairFace, a novel and balanced face dataset comprising 14,400 images representing geographical diversity of India. Images were sourced ethically from Wikimedia Commons and open-license web repositories and uniformly balanced across states and gender. Using IndicFairFace, we quantify intra-national geographical bias in prominent CLIP-based VLMs and reduce it using post-hoc Iterative Nullspace Projection debiasing approach. We also show that the adopted debiasing approach does not adversely impact the existing embedding space as the average drop in retrieval accuracy on benchmark datasets is less than 1.5 percent. Our work establishes IndicFairFace as the first benchmark to study geographical bias in VLMs for the Indian context.