# BERT Embeddings for Automatic Readability Assessment

arXiv: https://arxiv.org/abs/2106.07935 · HTML: https://mlsanity.com/paper/2106.07935/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 43 references that are in the map.
- First posted on arXiv 15 Jun 2021.

## Builds on (most cited first, 16 of 16)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [A Primer in BERTology: What we know about how BERT works](https://mlsanity.com/paper/2002.12327/) (2020)
- [What do you learn from context? Probing for sentence structure in contextualized word representations](https://mlsanity.com/paper/1905.06316/) (2019)
- [Linguistic Knowledge and Transferability of Contextual Representations](https://mlsanity.com/paper/1903.08855/) (2019)
- [What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language models](https://mlsanity.com/paper/1907.13528/) (2019)
- [Text Readability Assessment for Second Language Learners](https://mlsanity.com/paper/1906.07580/) (2019)
- [Supervised and Unsupervised Neural Approaches to Text Readability](https://mlsanity.com/paper/1907.11779/) (2019)
- [Linguistic Features for Readability Assessment](https://mlsanity.com/paper/2006.00377/) (2020)
- [Trends, Limitations and Open Challenges in Automatic Readability Assessment Research](https://mlsanity.com/paper/2105.00973/) (2021)
- [Evaluating Language Model Finetuning Techniques for Low-resource Languages](https://mlsanity.com/paper/1907.00409/) (2019)
- [Application of Lexical Features Towards Improvement of Filipino Readability Identification of Children's Literature](https://mlsanity.com/paper/2101.10537/) (2021)
- [Readability-based Sentence Ranking for Evaluating Text Simplification](https://mlsanity.com/paper/1603.06009/) (2016)
- [A Resource for Studying Chatino Verbal Morphology](https://mlsanity.com/paper/2004.02083/) (2020)
- [A Simple Post-Processing Technique for Improving Readability Assessment of Texts using Word Mover's Distance](https://mlsanity.com/paper/2103.07277/) (2021)

## Built on by (most cited first, 19 of 19)

- [CEFR-Based Sentence Difficulty Annotation and Assessment](https://mlsanity.com/paper/2210.11766/) (2022)
- [A Baseline Readability Model for Cebuano](https://mlsanity.com/paper/2203.17225/) (2022)
- [Automatic Readability Assessment for Closely Related Languages](https://mlsanity.com/paper/2305.13478/) (2023)
- [Prompt-based Learning for Text Readability Assessment](https://mlsanity.com/paper/2302.13139/) (2023)
- [Automatic Readability Assessment of German Sentences with Transformer Ensembles](https://mlsanity.com/paper/2209.04299/) (2022)
- [Standardize: Aligning Language Models with Expert-Defined Standards for Content Generation](https://mlsanity.com/paper/2402.12593/) (2024)
- [An Open Multilingual System for Scoring Readability of Wikipedia](https://mlsanity.com/paper/2406.01835/) (2024)
- [Ace-CEFR -- A Dataset for Automated Evaluation of the Linguistic Difficulty of Conversational Texts for LLM Applications](https://mlsanity.com/paper/2506.14046/) (2025)
- [Zero-shot Large Language Models for Automatic Readability Assessment](https://mlsanity.com/paper/2604.24470/) (2026)
- [Rethinking the Role of Text Complexity in Language Model Pretraining](https://mlsanity.com/paper/2509.16551/) (2025)
- [Large Language Models for Difficulty Estimation of Foreign Language Content with Application to Language Learning](https://mlsanity.com/paper/2309.05142/) (2023)
- [Gaze-Driven Sentence Simplification for Language Learners: Enhancing Comprehension and Readability](https://mlsanity.com/paper/2310.00355/) (2023)
- [Difficulty Estimation and Simplification of French Text Using LLMs](https://mlsanity.com/paper/2407.18061/) (2024)
- [Text-to-Distribution Prediction with Quantile Tokens and Neighbor Context](https://mlsanity.com/paper/2604.20216/) (2026)
- [Exploring Linguistic Features for Turkish Text Readability](https://mlsanity.com/paper/2306.03774/) (2023)
- [Cross-corpus Readability Compatibility Assessment for English Texts](https://mlsanity.com/paper/2306.09704/) (2023)
- [Hierarchical Ranking Neural Network for Long Document Readability Assessment](https://mlsanity.com/paper/2511.21473/) (2025)
- [From Critic to Confidence: PPO for Language-Based Quantitative Prediction with Confidence Estimation](https://mlsanity.com/paper/2607.12687/) (2026)
- [Analyzing Traditional and Neural Approaches to Multilingual Readability Assessment](https://mlsanity.com/paper/2609.10792/) (2026)

## Abstract (arXiv)

Automatic readability assessment (ARA) is the task of evaluating the level of ease or difficulty of text documents for a target audience. For researchers, one of the many open problems in the field is to make such models trained for the task show efficacy even for low-resource languages. In this study, we propose an alternative way of utilizing the information-rich embeddings of BERT models with handcrafted linguistic features through a combined method for readability assessment. Results show that the proposed method outperforms classical approaches in readability assessment using English and Filipino datasets, obtaining as high as 12.4% increase in F1 performance. We also show that the general information encoded in BERT embeddings can be used as a substitute feature set for low-resource languages like Filipino with limited semantic and syntactic NLP tools to explicitly extract feature values for the task.