# Towards Responsible Natural Language Annotation for the Varieties of Arabic

arXiv: https://arxiv.org/abs/2203.09597 · HTML: https://mlsanity.com/paper/2203.09597/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 40 references that are in the map.
- First posted on arXiv 17 Mar 2022.

## Builds on (most cited first, 11 of 11)

- [Model Cards for Model Reporting](https://mlsanity.com/paper/1810.03993/) (2018)
- [Semantics derived automatically from language corpora contain human-like biases](https://mlsanity.com/paper/1608.07187/) (2016)
- [Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarks](https://mlsanity.com/paper/2103.14749/) (2021)
- [Persistent Anti-Muslim Bias in Large Language Models](https://mlsanity.com/paper/2101.05783/) (2021)
- [Does Object Recognition Work for Everyone?](https://mlsanity.com/paper/1906.02659/) (2019)
- [Participatory Problem Formulation for Fairer Machine Learning Through Community Based System Dynamics](https://mlsanity.com/paper/2005.07572/) (2020)
- [Global AI Ethics: A Review of the Social Impacts and Ethical Implications of Artificial Intelligence](https://mlsanity.com/paper/1907.07892/) (2019)
- [Fairness On The Ground: Applying Algorithmic Fairness Approaches to Production Systems](https://mlsanity.com/paper/2103.06172/) (2021)
- [Institutionalising Ethics in AI through Broader Impact Requirements](https://mlsanity.com/paper/2106.11039/) (2021)
- [Reducing malicious use of synthetic media research: Considerations and potential release practices for machine learning](https://mlsanity.com/paper/1907.11274/) (2019)
- [Non-portability of Algorithmic Fairness in India](https://mlsanity.com/paper/2012.03659/) (2020)

## Built on by (most cited first, 14 of 14)

- [Lost in Translation: Large Language Models in Non-English Content Analysis](https://mlsanity.com/paper/2306.07377/) (2023)
- [ALDi: Quantifying the Arabic Level of Dialectness of Text](https://mlsanity.com/paper/2310.13747/) (2023)
- [Which Prompts Make The Difference? Data Prioritization For Efficient Human LLM Evaluation](https://mlsanity.com/paper/2310.14424/) (2023)
- [ChatGPT Rates Natural Language Explanation Quality Like Humans: But on Which Scales?](https://mlsanity.com/paper/2403.17368/) (2024)
- [AL-QASIDA: Analyzing LLM Quality and Accuracy Systematically in Dialectal Arabic](https://mlsanity.com/paper/2412.04193/) (2024)
- [LLM Alignment for the Arabs: A Homogenous Culture or Diverse Ones?](https://mlsanity.com/paper/2503.15003/) (2025)
- [Estimating the Level of Dialectness Predicts Interannotator Agreement in Multi-dialect Arabic Datasets](https://mlsanity.com/paper/2405.11282/) (2024)
- [Toxic language detection: a systematic review of Arabic datasets](https://mlsanity.com/paper/2312.07228/) (2023)
- [Revisiting Common Assumptions about Arabic Dialects in NLP](https://mlsanity.com/paper/2505.21816/) (2025)
- [DialUp! Modeling the Language Continuum by Adapting Models to Dialects and Dialects to Models](https://mlsanity.com/paper/2501.16581/) (2025)
- [AlexandriaX 2026: The First Shared Task on Dialectal Arabic Machine Translation](https://mlsanity.com/paper/2609.22796/) (2026)
- [Navigating Dialectal Bias and Ethical Complexities in Levantine Arabic Hate Speech Detection](https://mlsanity.com/paper/2412.10991/) (2024)
- [Maastricht University at AMIYA: Adapting LLMs for Dialectal Arabic using Fine-tuning and MBR Decoding](https://mlsanity.com/paper/2602.09703/) (2026)
- [SSDAU: Structured Semantic Data Augmentation for Joint Entity and Relation Extraction](https://mlsanity.com/paper/2605.23440/) (2026)

## Abstract (arXiv)

When building NLP models, there is a tendency to aim for broader coverage, often overlooking cultural and (socio)linguistic nuance. In this position paper, we make the case for care and attention to such nuances, particularly in dataset annotation, as well as the inclusion of cultural and linguistic expertise in the process. We present a playbook for responsible dataset creation for polyglossic, multidialectal languages. This work is informed by a study on Arabic annotation of social media content.