# Indian-BhED: A Dataset for Measuring India-Centric Biases in Large Language Models

arXiv: https://arxiv.org/abs/2309.08573 · HTML: https://mlsanity.com/paper/2309.08573/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 71 references that are in the map.
- First posted on arXiv 15 Sep 2023.

## Builds on (most cited first, 29 of 29)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings](https://mlsanity.com/paper/1607.06520/) (2016)
- [Language (Technology) is Power: A Critical Survey of "Bias" in NLP](https://mlsanity.com/paper/2005.14050/) (2020)
- [StereoSet: Measuring stereotypical bias in pretrained language models](https://mlsanity.com/paper/2004.09456/) (2020)
- [Whose Opinions Do Language Models Reflect?](https://mlsanity.com/paper/2303.17548/) (2023)
- [CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models](https://mlsanity.com/paper/2010.00133/) (2020)
- [RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignment](https://mlsanity.com/paper/2304.06767/) (2023)
- [Mitigating Gender Bias in Natural Language Processing: Literature Review](https://mlsanity.com/paper/1906.08976/) (2019)
- [From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP Models](https://mlsanity.com/paper/2305.08283/) (2023)
- [Racial Bias in Hate Speech and Abusive Language Detection Datasets](https://mlsanity.com/paper/1905.12516/) (2019)
- ["I'm sorry to hear that": Finding New Biases in Language Models with a Holistic Descriptor Dataset](https://mlsanity.com/paper/2205.09209/) (2022)
- [Counterfactual Fairness in Text Classification through Robustness](https://mlsanity.com/paper/1809.10610/) (2018)
- [Bias Out-of-the-Box: An Empirical Analysis of Intersectional Occupational Biases in Popular Generative Language Models](https://mlsanity.com/paper/2102.04130/) (2021)
- [Re-imagining Algorithmic Fairness in India and Beyond](https://mlsanity.com/paper/2101.09995/) (2021)
- [A Prompt Array Keeps the Bias Away: Debiasing Vision-Language Models with Adversarial Learning](https://mlsanity.com/paper/2203.11933/) (2022)
- [Unmasking the Mask -- Evaluating Social Biases in Masked Language Models](https://mlsanity.com/paper/2104.07496/) (2021)
- [Re-contextualizing Fairness in NLP: The Case of India](https://mlsanity.com/paper/2209.12226/) (2022)
- [Decoder-Only or Encoder-Decoder? Interpreting Language Model as a Regularized Encoder-Decoder](https://mlsanity.com/paper/2304.04052/) (2023)
- [A Trip Towards Fairness: Bias and De-Biasing in Large Language Models](https://mlsanity.com/paper/2305.13862/) (2023)
- [On Evaluating and Mitigating Gender Biases in Multilingual Settings](https://mlsanity.com/paper/2307.01503/) (2023)
- [Socially Aware Bias Measurements for Hindi Language Representations](https://mlsanity.com/paper/2110.07871/) (2021)
- [Unveiling Gender Bias in Terms of Profession Across LLMs: Analyzing and Addressing Sociological Implications](https://mlsanity.com/paper/2307.09162/) (2023)
- [Building Socio-culturally Inclusive Stereotype Resources with Community Engagement](https://mlsanity.com/paper/2307.10514/) (2023)
- [Hate Speech Detection from Code-mixed Hindi-English Tweets Using Deep Learning Models](https://mlsanity.com/paper/1811.05145/) (2018)
- ["Short is the Road that Leads from Fear to Hate": Fear Speech in Indian WhatsApp Groups](https://mlsanity.com/paper/2102.03870/) (2021)
- [Mitigating Gender Stereotypes in Hindi and Marathi](https://mlsanity.com/paper/2205.05901/) (2022)
- [Explaining Caste-based Digital Divide in India](https://mlsanity.com/paper/2106.15917/) (2021)

## Built on by (most cited first, 18 of 18)

- [Aya Expanse: Combining Research Breakthroughs for a New Multilingual Frontier](https://mlsanity.com/paper/2412.04261/) (2024)
- [Aya 23: Open Weight Releases to Further Multilingual Progress](https://mlsanity.com/paper/2405.15032/) (2024)
- [RLHF Can Speak Many Languages: Unlocking Multilingual Preference Optimization for LLMs](https://mlsanity.com/paper/2407.02552/) (2024)
- ['Simulacrum of Stories': Examining Large Language Models as Qualitative Research Participants](https://mlsanity.com/paper/2409.19430/) (2024)
- [DECASTE: Unveiling Caste Stereotypes in Large Language Models through Multi-Dimensional Bias Analysis](https://mlsanity.com/paper/2505.14971/) (2025)
- [TALES: A Taxonomy and Analysis of Cultural Representations in LLM-generated Stories](https://mlsanity.com/paper/2511.21322/) (2025)
- [BharatBBQ: A Multilingual Bias Benchmark for Question Answering in the Indian Context](https://mlsanity.com/paper/2508.07090/) (2025)
- [GIEBench: Towards Holistic Evaluation of Group Identity-based Empathy for Large Language Models](https://mlsanity.com/paper/2406.14903/) (2024)
- [Omissive Bias in Religious Representation: Benchmarking LLM Answers to Everyday Ethical Decision-making](https://mlsanity.com/paper/2605.24319/) (2026)
- [Recourse for reclamation: Chatting with generative language models](https://mlsanity.com/paper/2403.14467/) (2024)
- [CALMA: A Process for Deriving Context-aligned Axes for Language Model Alignment](https://mlsanity.com/paper/2507.09060/) (2025)
- [The Value of Gen-AI Conversations: A bottom-up Framework for AI Value Alignment](https://mlsanity.com/paper/2507.21091/) (2025)
- [Compounding Disadvantage: Auditing Intersectional Bias in LLM-Generated Explanations Across Indian and American STEM Education](https://mlsanity.com/paper/2601.14506/) (2026)
- [IndicFairFace: Balanced Indian Face Dataset for Auditing and Mitigating Geographical Bias in Vision-Language Models](https://mlsanity.com/paper/2602.12659/) (2026)
- [COMPASS: COntinual Multilingual PEFT with Adaptive Semantic Sampling](https://mlsanity.com/paper/2604.20720/) (2026)
- [What Do People Actually Want From AI? Mapping Preference Plurality](https://mlsanity.com/paper/2606.06674/) (2026)
- [Safety Alignment Illusion: The Cross-Lingual Safety Gap in LLMs](https://mlsanity.com/paper/2608.18131/) (2026)
- [RupeeBias: Auditing Demographic Bias in Indian Economic Guidance from Large Language Models](https://mlsanity.com/paper/2609.31245/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs), now used daily by millions, can encode societal biases, exposing their users to representational harms. A large body of scholarship on LLM bias exists but it predominantly adopts a Western-centric frame and attends comparatively less to bias levels and potential harms in the Global South. In this paper, we quantify stereotypical bias in popular LLMs according to an Indian-centric frame through Indian-BhED, a first of its kind dataset, containing stereotypical and anti-stereotypical examples in the context of caste and religious stereotypes in India. We find that the majority of LLMs tested have a strong propensity to output stereotypes in the Indian context, especially when compared to axes of bias traditionally studied in the Western context, such as gender and race. Notably, we find that GPT-2, GPT-2 Large, and GPT 3.5 have a particularly high propensity for preferring stereotypical outputs as a percent of all sentences for the axes of caste (63-79%) and religion (69-72%). We finally investigate potential causes for such harmful behaviour in LLMs, and posit intervention techniques to reduce both stereotypical and anti-stereotypical biases. The findings of this work highlight the need for including more diverse voices when researching fairness in AI and evaluating LLMs.