# When Tom Eats Kimchi: Evaluating Cultural Bias of Multimodal Large Language Models in Cultural Mixture Contexts

arXiv: https://arxiv.org/abs/2503.16826 · HTML: https://mlsanity.com/paper/2503.16826/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 8 of its 15 references that are in the map.
- First posted on arXiv 21 Mar 2025.

## Placed in

- Multimodal LLMs / vision-language models / [VLM benchmarks and visual reasoning](https://mlsanity.com/mllm/vlm-benchmarks-and-visual-reasoning/)

## Builds on (most cited first, 8 of 8)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Towards Measuring and Modeling "Culture" in LLMs: A Survey](https://mlsanity.com/paper/2403.15412/) (2024)
- [Benchmarking Vision Language Models for Cultural Understanding](https://mlsanity.com/paper/2407.10920/) (2024)
- [Examining Gender and Racial Bias in Large Vision-Language Models Using a Novel Dataset of Parallel Images](https://mlsanity.com/paper/2402.05779/) (2024)
- [Gender and Racial Bias in Visual Question Answering Datasets](https://mlsanity.com/paper/2205.08148/) (2022)
- [Uncovering Bias in Large Vision-Language Models at Scale with Counterfactuals](https://mlsanity.com/paper/2405.20152/) (2024)
- [Evaluating Visual and Cultural Interpretation: The K-Viscuit Benchmark with Human-VLM Collaboration](https://mlsanity.com/paper/2406.16469/) (2024)
- [Semi-Truths: A Large-Scale Dataset of AI-Augmented Images for Evaluating Robustness of AI-Generated Image detectors](https://mlsanity.com/paper/2411.07472/) (2024)

## Built on by (most cited first, 3 of 3)

- [Interpretable Debiasing of Vision-Language Models for Social Fairness](https://mlsanity.com/paper/2602.24014/) (2026)
- [XCR-Bench: Benchmarking Cross-Cultural Reasoning in LLMs via Culture-Specific Items and Hall's Triad](https://mlsanity.com/paper/2601.14063/) (2026)
- [A Stereotype Content Analysis on Color-related Social Bias in Large Vision Language Models](https://mlsanity.com/paper/2505.20901/) (2025)

## Abstract (arXiv)

In a highly globalized world, it is important for multi-modal large language models (MLLMs) to recognize and respond correctly to mixed-cultural inputs. For example, a model should correctly identify kimchi (Korean food) in an image both when an Asian woman is eating it, as well as an African man is eating it. However, current MLLMs show an over-reliance on the visual features of the person, leading to misclassification of the entities. To examine the robustness of MLLMs to different ethnicity, we introduce MixCuBe, a cross-cultural bias benchmark, and study elements from five countries and four ethnicities. Our findings reveal that MLLMs achieve both higher accuracy and lower sensitivity to such perturbation for high-resource cultures, but not for low-resource cultures. GPT-4o, the best-performing model overall, shows up to 58% difference in accuracy between the original and perturbed cultural settings in low-resource cultures. Our dataset is publicly available at: https://huggingface.co/datasets/kyawyethu/MixCuBe.