mlsanityData to 7 Oct 2026

BERT Embeddings for Automatic Readability Assessment

arXiv 2106.07935 ↗First posted 15 Jun 2021cs.CLcited by 19

Not on the map. Why?

Abstract (arXiv)

Automatic readability assessment (ARA) is the task of evaluating the level of ease or difficulty of text documents for a target audience. For researchers, one of the many open problems in the field is to make such models trained for the task show efficacy even for low-resource languages. In this study, we propose an alternative way of utilizing the information-rich embeddings of BERT models with handcrafted linguistic features through a combined method for readability assessment. Results show that the proposed method outperforms classical approaches in readability assessment using English and Filipino datasets, obtaining as high as 12.4% increase in F1 performance. We also show that the general information encoded in BERT embeddings can be used as a substitute feature set for low-resource languages like Filipino with limited semantic and syntactic NLP tools to explicitly extract feature values for the task.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Builds on 16 of its 43 references that are in the map. First posted on arXiv 15 Jun 2021.

Builds on

16 of 43 references.

References · 16

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  4. 2020A Primer in BERTology: What we know about how BERT workscited by 807
  5. 2019What do you learn from context? Probing for sentence structure in contextualized word representationscited by 436
  6. 2019Linguistic Knowledge and Transferability of Contextual Representationscited by 382
  7. 2019What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language modelscited by 317
  8. 2019Text Readability Assessment for Second Language Learnerscited by 54
  9. 2019Supervised and Unsupervised Neural Approaches to Text Readabilitycited by 45
  10. 2020Linguistic Features for Readability Assessmentcited by 35
  11. 2021Trends, Limitations and Open Challenges in Automatic Readability Assessment Researchcited by 28
  12. 2019Evaluating Language Model Finetuning Techniques for Low-resource Languagescited by 12
4 more
  1. 2021Application of Lexical Features Towards Improvement of Filipino Readability Identification of Children's Literaturecited by 6
  2. 2016Readability-based Sentence Ranking for Evaluating Text Simplificationcited by 5
  3. 2020A Resource for Studying Chatino Verbal Morphologycited by 2
  4. 2021A Simple Post-Processing Technique for Improving Readability Assessment of Texts using Word Mover's Distancecited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 6; 2024: 3; 2025: 3; 2026: 4.

19 papers.

Sort:

By area:

  1. 2022CEFR-Based Sentence Difficulty Annotation and Assessmentcited by 28
  2. 2022A Baseline Readability Model for Cebuanocited by 10
  3. 2023Automatic Readability Assessment for Closely Related Languagescited by 9
  4. 2023Prompt-based Learning for Text Readability Assessmentcited by 8
  5. 2022Automatic Readability Assessment of German Sentences with Transformer Ensemblescited by 7
  6. 2024Standardize: Aligning Language Models with Expert-Defined Standards for Content Generationcited by 7
  7. 2024An Open Multilingual System for Scoring Readability of Wikipediacited by 7
  8. 2025Ace-CEFR -- A Dataset for Automated Evaluation of the Linguistic Difficulty of Conversational Texts for LLM Applicationscited by 4
  9. 2026Zero-shot Large Language Models for Automatic Readability Assessmentcited by 3
  10. 2025Rethinking the Role of Text Complexity in Language Model Pretrainingcited by 2
  11. 2023Large Language Models for Difficulty Estimation of Foreign Language Content with Application to Language Learningcited by 1
  12. 2023Gaze-Driven Sentence Simplification for Language Learners: Enhancing Comprehension and Readabilitycited by 1
7 more
  1. 2024Difficulty Estimation and Simplification of French Text Using LLMscited by 1
  2. 2023Exploring Linguistic Features for Turkish Text Readabilitycited by 0
  3. 2023Cross-corpus Readability Compatibility Assessment for English Textscited by 0
  4. 2025Hierarchical Ranking Neural Network for Long Document Readability Assessmentcited by 0
  5. 2026Text-to-Distribution Prediction with Quantile Tokens and Neighbor Contextcited by 0
  6. 2026From Critic to Confidence: PPO for Language-Based Quantitative Prediction with Confidence Estimationcited by 0
  7. 2026Analyzing Traditional and Neural Approaches to Multilingual Readability Assessmentcited by 0