Considerations for Multilingual Wikipedia Research
Not on the map. Why?
Abstract (arXiv)
English Wikipedia has long been an important data source for much research and natural language machine learning modeling. The growth of non-English language editions of Wikipedia, greater computational resources, and calls for equity in the performance of language and multimodal models have led to the inclusion of many more language editions of Wikipedia in datasets and models. Building better multilingual and multimodal models requires more than just access to expanded datasets; it also requires a better understanding of what is in the data and how this content was generated. This paper seeks to provide some background to help researchers think about what differences might arise between different language editions of Wikipedia and how that might affect their models. It details three major ways in which content differences between language editions arise (local context, community and governance, and technology) and recommendations for good practices when using multilingual and multimodal data for research and modeling.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 31 of its 69 references that are in the map. First posted on arXiv 5 Apr 2022.
Builds on
31 of 69 references.
References · 31
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
- 2016Pointer Sentinel Mixture Modelscited by 2,992
- 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
- 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,544
- 2018Datasheets for Datasetscited by 1,371
- 2022Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Modelcited by 557
- 2021Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpuscited by 481
- 2021Multimodal datasets: misogyny, pornography, and malignant stereotypescited by 326
- 2021WIT: Wikipedia-based Image Text Dataset for Multimodal Multilingual Machine Learningcited by 285
- 2020Data and its (dis)contents: A survey of dataset development and use in machine learning researchcited by 259
- 2021Quality at a Glance: An Audit of Web-Crawled Multilingual Datasetscited by 241
19 more
- 2019WikiMatrix: Mining 135M Parallel Sentences in 1620 Language Pairs from Wikipediacited by 221
- 2020Are All Languages Created Equal in Multilingual BERT?cited by 206
- 2020Participatory Research for Low-resourced Machine Translation: A Case Study in African Languagescited by 159
- 2019Corpora Generation for Grammatical Error Correctioncited by 53
- 2018WikiConv: A Corpus of the Complete Conversational History of a Large Online Collaborative Communitycited by 16
- 2018Why the World Reads Wikipedia: Beyond English Speakerscited by 15
- 2020Controlled Analyses of Social Biases in Wikipedia Bioscited by 13
- 2017The Wisdom of Polarized Crowdscited by 9
- 2015Understanding Editing Behaviors in Multilingual Wikipediacited by 6
- 2019Not at Home on the Range: Peer Production and the Urban/Rural Dividecited by 6
- 2021Language-agnostic Topic Classification for Wikipediacited by 6
- 2022Wikipedia Reader Navigation: When Synthetic Data Is Enoughcited by 6
- 2020Crosslingual Topic Modeling with WikiPDAcited by 5
- 2019Wikipedia Cultural Diversity Dataset: A Complete Cartography for 300 Language Editionscited by 4
- 2020Global gender differences in Wikipedia readershipcited by 4
- 2020WikiHist.html: English Wikipedia's Full Revision History in HTML Formatcited by 3
- 2018Wikipedia Text Reuse: Within and Withoutcited by 2
- 2020Analyzing Wikidata Transclusion on English Wikipediacited by 2
- 2019Detecting and Gauging Impact on Wikipedia Page Viewscited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.