# An Open Multilingual System for Scoring Readability of Wikipedia

arXiv: https://arxiv.org/abs/2406.01835 · HTML: https://mlsanity.com/paper/2406.01835/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 86 references that are in the map.
- First posted on arXiv 3 Jun 2024.

## Builds on (most cited first, 26 of 26)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [The Pile: An 800GB Dataset of Diverse Text for Language Modeling](https://mlsanity.com/paper/2101.00027/) (2020)
- [How multilingual is Multilingual BERT?](https://mlsanity.com/paper/1906.01502/) (2019)
- [Quality at a Glance: An Audit of Web-Crawled Multilingual Datasets](https://mlsanity.com/paper/2103.12028/) (2021)
- [Are All Languages Created Equal in Multilingual BERT?](https://mlsanity.com/paper/2005.09093/) (2020)
- [Sentence Simplification with Deep Reinforcement Learning](https://mlsanity.com/paper/1703.10931/) (2017)
- [Neural CRF Model for Sentence Alignment in Text Simplification](https://mlsanity.com/paper/2005.02324/) (2020)
- [Text Readability Assessment for Second Language Learners](https://mlsanity.com/paper/1906.07580/) (2019)
- [Supervised and Unsupervised Neural Approaches to Text Readability](https://mlsanity.com/paper/1907.11779/) (2019)
- [Linguistic Features for Readability Assessment](https://mlsanity.com/paper/2006.00377/) (2020)
- [CatBoost: unbiased boosting with categorical features](https://mlsanity.com/paper/1706.09516/) (2017)
- [Trends, Limitations and Open Challenges in Automatic Readability Assessment Research](https://mlsanity.com/paper/2105.00973/) (2021)
- [Document-Level Text Simplification: Dataset, Criteria and Baseline](https://mlsanity.com/paper/2110.05071/) (2021)
- [BERT Embeddings for Automatic Readability Assessment](https://mlsanity.com/paper/2106.07935/) (2021)
- [Why the World Reads Wikipedia: Beyond English Speakers](https://mlsanity.com/paper/1812.00474/) (2018)
- [A Neural Pairwise Ranking Model for Readability Assessment](https://mlsanity.com/paper/2203.07450/) (2022)
- [SWiPE: A Dataset for Document-Level Simplification of Wikipedia Pages](https://mlsanity.com/paper/2305.19204/) (2023)
- [A Baseline Readability Model for Cebuano](https://mlsanity.com/paper/2203.17225/) (2022)
- [mLongT5: A Multilingual and Efficient Text-To-Text Transformer for Longer Sequences](https://mlsanity.com/paper/2305.11129/) (2023)
- [Automatic Readability Assessment for Closely Related Languages](https://mlsanity.com/paper/2305.13478/) (2023)
- [Prompt-based Learning for Text Readability Assessment](https://mlsanity.com/paper/2302.13139/) (2023)
- [Text as Environment: A Deep Reinforcement Learning Text Readability Assessment Model](https://mlsanity.com/paper/1912.05957/) (2019)
- [Automatic Readability Assessment of German Sentences with Transformer Ensembles](https://mlsanity.com/paper/2209.04299/) (2022)
- [Why We Read Wikipedia](https://mlsanity.com/paper/1702.05379/) (2017)
- [Global gender differences in Wikipedia readership](https://mlsanity.com/paper/2007.10403/) (2020)
- [WikiHist.html: English Wikipedia's Full Revision History in HTML Format](https://mlsanity.com/paper/2001.10256/) (2020)

## Built on by (most cited first, 7 of 7)

- [MO-GRPO: Mitigating Reward Hacking of Group Relative Policy Optimization on Multi-Objective Problems](https://mlsanity.com/paper/2509.22047/) (2025)
- [Wikipedia in the Era of LLMs: Evolution and Risks](https://mlsanity.com/paper/2503.02879/) (2025)
- [MULTITAT: Benchmarking Multilingual Table-and-Text Question Answering](https://mlsanity.com/paper/2502.17253/) (2025)
- [Multilingual Reference Need Assessment System for Wikipedia](https://mlsanity.com/paper/2603.17146/) (2026)
- [Gatsby Without the 'E': Crafting Lipograms with LLMs](https://mlsanity.com/paper/2505.20501/) (2025)
- [Hidden Persuasion: Detecting Manipulative Narratives on Social Media During the 2022 Russian Invasion of Ukraine](https://mlsanity.com/paper/2505.24028/) (2025)
- [An End-to-End Ukrainian RAG for Local Deployment. Optimized Hybrid Search and Lightweight Generation](https://mlsanity.com/paper/2604.22095/) (2026)

## Abstract (arXiv)

With over 60M articles, Wikipedia has become the largest platform for open and freely accessible knowledge. While it has more than 15B monthly visits, its content is believed to be inaccessible to many readers due to the lack of readability of its text. However, previous investigations of the readability of Wikipedia have been restricted to English only, and there are currently no systems supporting the automatic readability assessment of the 300+ languages in Wikipedia. To bridge this gap, we develop a multilingual model to score the readability of Wikipedia articles. To train and evaluate this model, we create a novel multilingual dataset spanning 14 languages, by matching articles from Wikipedia to simplified Wikipedia and online children encyclopedias. We show that our model performs well in a zero-shot scenario, yielding a ranking accuracy of more than 80% across 14 languages and improving upon previous benchmarks. These results demonstrate the applicability of the model at scale for languages in which there is no ground-truth data available for model fine-tuning. Furthermore, we provide the first overview on the state of readability in Wikipedia beyond English.