# A Neural Pairwise Ranking Model for Readability Assessment

arXiv: https://arxiv.org/abs/2203.07450 · HTML: https://mlsanity.com/paper/2203.07450/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 66 references that are in the map.
- First posted on arXiv 14 Mar 2022.

## Builds on (most cited first, 16 of 16)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [Text Readability Assessment for Second Language Learners](https://mlsanity.com/paper/1906.07580/) (2019)
- [Supervised and Unsupervised Neural Approaches to Text Readability](https://mlsanity.com/paper/1907.11779/) (2019)
- [Learning-to-Rank with BERT in TF-Ranking](https://mlsanity.com/paper/2004.08476/) (2020)
- [Pushing on Text Readability Assessment: A Transformer Meets Handcrafted Linguistic Features](https://mlsanity.com/paper/2109.12258/) (2021)
- [Linguistic Features for Readability Assessment](https://mlsanity.com/paper/2006.00377/) (2020)
- [TF-Ranking: Scalable TensorFlow Library for Learning-to-Rank](https://mlsanity.com/paper/1812.00073/) (2018)
- [Controlling Text Complexity in Neural Machine Translation](https://mlsanity.com/paper/1911.00835/) (2019)
- [Learning Groupwise Multivariate Scoring Functions Using Deep Neural Networks](https://mlsanity.com/paper/1811.04415/) (2018)
- [ReadNet: A Hierarchical Transformer Framework for Web Article Readability Analysis](https://mlsanity.com/paper/2103.04083/) (2021)
- [Language Modeling by Clustering with Word Embeddings for Text Readability Assessment](https://mlsanity.com/paper/1709.01888/) (2017)
- [Text as Environment: A Deep Reinforcement Learning Text Readability Assessment Model](https://mlsanity.com/paper/1912.05957/) (2019)
- [RankDCG: Rank-Ordering Evaluation Measure](https://mlsanity.com/paper/1803.00719/) (2018)

## Built on by (most cited first, 14 of 14)

- [Simplicity Level Estimate (SLE): A Learned Reference-Less Metric for Sentence Simplification](https://mlsanity.com/paper/2310.08170/) (2023)
- [ULTRA: Unleash LLMs' Potential for Event Argument Extraction through Hierarchical Modeling and Pair-wise Self-Refinement](https://mlsanity.com/paper/2401.13218/) (2024)
- [Learning to Paraphrase Sentences to Different Complexity Levels](https://mlsanity.com/paper/2308.02226/) (2023)
- [Automatic Readability Assessment for Closely Related Languages](https://mlsanity.com/paper/2305.13478/) (2023)
- [Prompt-based Learning for Text Readability Assessment](https://mlsanity.com/paper/2302.13139/) (2023)
- [A Unified Neural Network Model for Readability Assessment with Feature Projection and Length-Balanced Loss](https://mlsanity.com/paper/2210.10305/) (2022)
- [An Open Multilingual System for Scoring Readability of Wikipedia](https://mlsanity.com/paper/2406.01835/) (2024)
- [ImpScore: A Learnable Metric For Quantifying The Implicitness Level of Sentence](https://mlsanity.com/paper/2411.05172/) (2024)
- [BasahaCorpus: An Expanded Linguistic Resource for Readability Assessment in Central Philippine Languages](https://mlsanity.com/paper/2310.11584/) (2023)
- [Strategies for Arabic Readability Modeling](https://mlsanity.com/paper/2407.03032/) (2024)
- [FPT: Feature Prompt Tuning for Few-shot Readability Assessment](https://mlsanity.com/paper/2404.02772/) (2024)
- [Cross-corpus Readability Compatibility Assessment for English Texts](https://mlsanity.com/paper/2306.09704/) (2023)
- [Text Simplification with Sentence Embeddings](https://mlsanity.com/paper/2510.24365/) (2025)
- [Hierarchical Ranking Neural Network for Long Document Readability Assessment](https://mlsanity.com/paper/2511.21473/) (2025)

## Abstract (arXiv)

Automatic Readability Assessment (ARA), the task of assigning a reading level to a text, is traditionally treated as a classification problem in NLP research. In this paper, we propose the first neural, pairwise ranking approach to ARA and compare it with existing classification, regression, and (non-neural) ranking methods. We establish the performance of our model by conducting experiments with three English, one French and one Spanish datasets. We demonstrate that our approach performs well in monolingual single/cross corpus testing scenarios and achieves a zero-shot cross-lingual ranking accuracy of over 80% for both French and Spanish when trained on English data. Additionally, we also release a new parallel bilingual readability dataset in English and French. To our knowledge, this paper proposes the first neural pairwise ranking model for ARA, and shows the first results of cross-lingual, zero-shot evaluation of ARA with neural models.