mlsanityData to 7 Oct 2026

What are Foundation Models Cooking in the Post-Soviet World?

arXiv 2502.18583 ↗First posted 25 Feb 2025cs.CLcited by 3

Not on the map. Why?

Abstract (arXiv)

The culture of the Post-Soviet states is complex, shaped by a turbulent history that continues to influence current events. In this study, we investigate the Post-Soviet cultural food knowledge of foundation models by constructing BORSch, a multimodal dataset encompassing 1147 and 823 dishes in the Russian and Ukrainian languages, centered around the Post-Soviet region. We demonstrate that leading models struggle to correctly identify the origins of dishes from Post-Soviet nations in both text-only and multimodal Question Answering (QA), instead over-predicting countries linked to the language the question is asked in. Through analysis of pretraining data, we show that these results can be explained by misleading dish-origin co-occurrences, along with linguistic phenomena such as Russian-Ukrainian code mixing. Finally, to move beyond QA-based assessments, we test models' abilities to produce accurate visual descriptions of dishes. The weak correlation between this task and QA suggests that QA alone may be insufficient as an evaluation of cultural understanding. To foster further research, we will make BORSch publicly available at https://github.com/alavrouk/BORSch.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 30 of its 62 references that are in the map. First posted on arXiv 25 Feb 2025.

Builds on

30 of 62 references.

References · 30

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  4. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  5. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  6. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  7. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  8. 2024Qwen2 Technical Reportcited by 2,076
  9. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,813
  10. 2024Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preferencecited by 1,151
  11. 2021Quality at a Glance: An Audit of Web-Crawled Multilingual Datasetscited by 241
  12. 2024Towards Measuring and Modeling "Culture" in LLMs: A Surveycited by 166
18 more
  1. 2024Political Compass or Spinning Arrow? Towards More Meaningful Evaluations for Values and Opinions in Large Language Modelscited by 162
  2. 2023Having Beer after Prayer? Measuring Cultural Bias in Large Language Modelscited by 153
  3. 2024GenAI Arena: An Open Evaluation Platform for Generative Modelscited by 81
  4. 2022Extracting Cultural Commonsense Knowledge at Scalecited by 76
  5. 2024How Well Do LLMs Represent Values Across Cultures? Empirical Analysis of LLM Responses Based on Hofstede Cultural Dimensionscited by 51
  6. 2023GeoDE: a Geographically Diverse Evaluation Dataset for Object Recognitioncited by 50
  7. 2024Understanding the Capabilities and Limitations of Large Language Models for Cultural Commonsensecited by 50
  8. 2024WorldCuisines: A Massive-Scale Benchmark for Multilingual and Multicultural Visual Question Answering on Global Cuisinescited by 39
  9. 2024Can LLM Generate Culturally Relevant Commonsense QA Data? Case Study in Indonesian and Sundanesecited by 36
  10. 2024CaLMQA: Exploring culturally specific long-form question answering across 23 languagescited by 31
  11. 2024Does Mapo Tofu Contain Coffee? Probing LLMs for Food-related Cultural Knowledgecited by 29
  12. 2024M5 -- A Diverse Benchmark to Assess the Performance of Large Multimodal Models Across Multilingual and Multicultural Vision-Language Taskscited by 27
  13. 2025On The Origin of Cultural Biases in Language Models: From Pre-training Data to Linguistic Phenomenacited by 21
  14. 2024Cultural Commonsense Knowledge for Intercultural Dialoguescited by 20
  15. 2024From Local Concepts to Universals: Evaluating the Multicultural Understanding of Vision-Language Modelscited by 14
  16. 2024Vision-Language Models under Cultural and Inclusive Considerationscited by 8
  17. 2024Attributing Culture-Conditioned Generations to Pretraining Corporacited by 8
  18. 2024Benchmarking Multimodal Models for Ukrainian Language Understanding Across Academic and Cultural Domainscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

  1. 2025CUS-QA: Local-Knowledge-Oriented Open-Ended Question Answering Datasetcited by 4
  2. 2025Camellia: Benchmarking Cultural Biases in LLMs for Asian Languagescited by 2
  3. 2026CultureMINE: Datasets and Methods for Improving the Cultural Capabilities of NLP Systemscited by 0