# The World Wide recipe: A community-centred framework for fine-grained data collection and regional bias operationalisation

arXiv: https://arxiv.org/abs/2406.09496 · HTML: https://mlsanity.com/paper/2406.09496/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 14 references that are in the map.
- First posted on arXiv 13 Jun 2024.

## Builds on (most cited first, 8 of 8)

- [CLIPScore: A Reference-free Evaluation Metric for Image Captioning](https://mlsanity.com/paper/2104.08718/) (2021)
- [XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Models](https://mlsanity.com/paper/2308.01263/) (2023)
- [Measurement and Fairness](https://mlsanity.com/paper/1912.05511/) (2019)
- [Easily Accessible Text-to-Image Generation Amplifies Demographic Stereotypes at Large Scale](https://mlsanity.com/paper/2211.03759/) (2022)
- [MasakhaNER: Named Entity Recognition for African Languages](https://mlsanity.com/paper/2103.11811/) (2021)
- [Power to the People? Opportunities and Challenges for Participatory AI](https://mlsanity.com/paper/2209.07572/) (2022)
- [The Participatory Turn in AI Design: Theoretical Foundations and the Current State of Practice](https://mlsanity.com/paper/2310.00907/) (2023)
- [This Thing Called Fairness: Disciplinary Confusion Realizing a Value in Technology](https://mlsanity.com/paper/1909.11869/) (2019)

## Built on by (most cited first, 7 of 7)

- [Survey of Cultural Awareness in Language Models: Text and Beyond](https://mlsanity.com/paper/2411.00860/) (2024)
- [Charting the Landscape of African NLP: Mapping Progress and Shaping the Road Ahead](https://mlsanity.com/paper/2505.21315/) (2025)
- [DRISHTIKON: A Multimodal Multilingual Benchmark for Testing Language Models' Understanding on Indian Culture](https://mlsanity.com/paper/2509.19274/) (2025)
- [World in a Frame: Understanding Culture Mixing as a New Challenge for Vision-Language Models](https://mlsanity.com/paper/2511.22787/) (2025)
- [The Consensus Trap: Dissecting Subjectivity and the "Ground Truth" Illusion in Data Annotation](https://mlsanity.com/paper/2602.11318/) (2026)
- [Going PLACES: Participatory Localized Red Teaming for Text-to-Image Safety in the Global South](https://mlsanity.com/paper/2605.19190/) (2026)
- [Eating for a Sustainable Planet: Personalized Sustainable Diet Recommendation via Constraint-Aware Decision-Making Modeling](https://mlsanity.com/paper/2608.24274/) (2026)

## Abstract (arXiv)

We introduce the World Wide recipe, which sets forth a framework for culturally aware and participatory data collection, and the resultant regionally diverse World Wide Dishes evaluation dataset. We also analyse bias operationalisation to highlight how current systems underperform across several dimensions: (in-)accuracy, (mis-)representation, and cultural (in-)sensitivity, with evidence from qualitative community-based observations and quantitative automated tools. We find that these T2I models generally do not produce quality outputs of dishes specific to various regions. This is true even for the US, which is typically considered more well-resourced in training data -- although the generation of US dishes does outperform that of the investigated African countries. The models demonstrate the propensity to produce inaccurate and culturally misrepresentative, flattening, and insensitive outputs. These representational biases have the potential to further reinforce stereotypes and disproportionately contribute to erasure based on region. The dataset and code are available at https://github.com/oxai/world-wide-dishes.