# SANSKRITI: A Comprehensive Benchmark for Evaluating Language Models' Knowledge of Indian Culture

arXiv: https://arxiv.org/abs/2506.15355 · HTML: https://mlsanity.com/paper/2506.15355/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 38 references that are in the map.
- First posted on arXiv 18 Jun 2025.

## Builds on (most cited first, 16 of 16)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Language (Technology) is Power: A Critical Survey of "Bias" in NLP](https://mlsanity.com/paper/2005.14050/) (2020)
- [TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languages](https://mlsanity.com/paper/2003.05002/) (2020)
- [Towards Measuring the Representation of Subjective Global Opinions in Language Models](https://mlsanity.com/paper/2306.16388/) (2023)
- [Style Transfer from Non-Parallel Text by Cross-Alignment](https://mlsanity.com/paper/1705.09655/) (2017)
- [CVQA: Culturally-diverse Multilingual Visual Question Answering Benchmark](https://mlsanity.com/paper/2406.05967/) (2024)
- [Cultural Alignment in Large Language Models: An Explanatory Analysis Based on Hofstede's Cultural Dimensions](https://mlsanity.com/paper/2309.12342/) (2023)
- [The Ghost in the Machine has an American accent: value conflict in GPT-3](https://mlsanity.com/paper/2203.07785/) (2022)
- [IndicGenBench: A Multilingual Benchmark to Evaluate Generation Capabilities of LLMs on Indic Languages](https://mlsanity.com/paper/2404.16816/) (2024)
- [CULTURE-GEN: Revealing Global Cultural Perception in Language Models through Natural Language Prompting](https://mlsanity.com/paper/2404.10199/) (2024)
- [A Survey of Multilingual Reasoning in Language Models](https://mlsanity.com/paper/2502.09457/) (2025)
- [DOSA: A Dataset of Social Artifacts from Different Indian Geographical Subcultures](https://mlsanity.com/paper/2403.14651/) (2024)
- [MedSumm: A Multimodal Approach to Summarizing Code-Mixed Hindi-English Clinical Queries](https://mlsanity.com/paper/2401.01596/) (2024)
- [LoFTI: Localization and Factuality Transfer to Indian Locales](https://mlsanity.com/paper/2407.11833/) (2024)

## Built on by (most cited first, 16 of 16)

- [DRISHTIKON: A Multimodal Multilingual Benchmark for Testing Language Models' Understanding on Indian Culture](https://mlsanity.com/paper/2509.19274/) (2025)
- [TALES: A Taxonomy and Analysis of Cultural Representations in LLM-generated Stories](https://mlsanity.com/paper/2511.21322/) (2025)
- [CLINIC: Evaluating Multilingual Trustworthiness in Language Models for Healthcare](https://mlsanity.com/paper/2512.11437/) (2025)
- [PARAM-1 BharatGen 2.9B Model](https://mlsanity.com/paper/2507.13390/) (2025)
- [CarePilot: A Multi-Agent Framework for Long-Horizon Computer Task Automation in Healthcare](https://mlsanity.com/paper/2603.24157/) (2026)
- [ParamBench: A Graduate-Level Benchmark for Evaluating LLM Understanding on Indic Subjects](https://mlsanity.com/paper/2508.16185/) (2025)
- [IndicParam: Benchmark to evaluate LLMs on low-resource Indic Languages](https://mlsanity.com/paper/2512.00333/) (2025)
- [Common to Whom? Regional Cultural Commonsense and LLM Bias in India](https://mlsanity.com/paper/2601.15550/) (2026)
- [ELR-1000: A Community-Generated Dataset for Endangered Indic Indigenous Languages](https://mlsanity.com/paper/2512.01077/) (2025)
- [BanglaSocialBench: A Benchmark for Evaluating Sociopragmatic and Cultural Alignment of LLMs in Bangladeshi Social Interaction](https://mlsanity.com/paper/2603.15949/) (2026)
- [BhashaSutra: A Task-Centric Unified Survey of Indian NLP Datasets, Corpora, and Resources](https://mlsanity.com/paper/2604.18423/) (2026)
- [When Meaning Travels: A Granular Lens on Hybrid-MoE's Role in Idiomatic Understanding for Language Models](https://mlsanity.com/paper/2606.01671/) (2026)
- [Characterizing Cultural Localization in AI-Generated Stories](https://mlsanity.com/paper/2606.14626/) (2026)
- [Benchmark-Based Comparative Assessment of Publicly Benchmarked Indian Foundation Models: A Capability and Evaluation-Maturity Framework](https://mlsanity.com/paper/2608.11891/) (2026)
- [CultureMINE: Datasets and Methods for Improving the Cultural Capabilities of NLP Systems](https://mlsanity.com/paper/2609.22494/) (2026)
- [In-Place Indic Tokenizer Expansion and Continual Pretraining of Nemotron-3-Nano-30B-A3B](https://mlsanity.com/paper/2610.15373/) (2026)

## Abstract (arXiv)

Language Models (LMs) are indispensable tools shaping modern workflows, but their global effectiveness depends on understanding local socio-cultural contexts. To address this, we introduce SANSKRITI, a benchmark designed to evaluate language models' comprehension of India's rich cultural diversity. Comprising 21,853 meticulously curated question-answer pairs spanning 28 states and 8 union territories, SANSKRITI is the largest dataset for testing Indian cultural knowledge. It covers sixteen key attributes of Indian culture: rituals and ceremonies, history, tourism, cuisine, dance and music, costume, language, art, festivals, religion, medicine, transport, sports, nightlife, and personalities, providing a comprehensive representation of India's cultural tapestry. We evaluate SANSKRITI on leading Large Language Models (LLMs), Indic Language Models (ILMs), and Small Language Models (SLMs), revealing significant disparities in their ability to handle culturally nuanced queries, with many models struggling in region-specific contexts. By offering an extensive, culturally rich, and diverse dataset, SANSKRITI sets a new standard for assessing and improving the cultural understanding of LMs.