# CEFR-Based Sentence Difficulty Annotation and Assessment

arXiv: https://arxiv.org/abs/2210.11766 · HTML: https://mlsanity.com/paper/2210.11766/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 72 references that are in the map.
- First posted on arXiv 21 Oct 2022.

## Builds on (most cited first, 19 of 19)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [Cross-lingual Language Model Pretraining](https://mlsanity.com/paper/1901.07291/) (2019)
- [Stanza: A Python Natural Language Processing Toolkit for Many Human Languages](https://mlsanity.com/paper/2003.07082/) (2020)
- [Sentence Simplification with Deep Reinforcement Learning](https://mlsanity.com/paper/1703.10931/) (2017)
- [Neural CRF Model for Sentence Alignment in Text Simplification](https://mlsanity.com/paper/2005.02324/) (2020)
- [Text Readability Assessment for Second Language Learners](https://mlsanity.com/paper/1906.07580/) (2019)
- [Supervised and Unsupervised Neural Approaches to Text Readability](https://mlsanity.com/paper/1907.11779/) (2019)
- [Linguistic Features for Readability Assessment](https://mlsanity.com/paper/2006.00377/) (2020)
- [Multi-Level Matching and Aggregation Network for Few-Shot Relation Classification](https://mlsanity.com/paper/1906.06678/) (2019)
- [BERT Embeddings for Automatic Readability Assessment](https://mlsanity.com/paper/2106.07935/) (2021)
- [Subjective Assessment of Text Complexity: A Dataset for German Language](https://mlsanity.com/paper/1904.07733/) (2019)
- [ReadNet: A Hierarchical Transformer Framework for Web Article Readability Analysis](https://mlsanity.com/paper/2103.04083/) (2021)
- [Explainable Prediction of Text Complexity: The Missing Preliminaries for Text Simplification](https://mlsanity.com/paper/2007.15823/) (2020)
- [Are pre-trained text representations useful for multilingual and multi-dimensional language proficiency modeling?](https://mlsanity.com/paper/2102.12971/) (2021)

## Built on by (most cited first, 28 of 28)

- [BOUQuET: dataset, Benchmark and Open initiative for Universal Quality Evaluation in Translation](https://mlsanity.com/paper/2502.04314/) (2025)
- [ReadMe++: Benchmarking Multilingual Language Models for Multi-Domain Readability Assessment](https://mlsanity.com/paper/2305.14463/) (2023)
- [Analysing Zero-Shot Readability-Controlled Sentence Simplification](https://mlsanity.com/paper/2409.20246/) (2024)
- [Learning to Paraphrase Sentences to Different Complexity Levels](https://mlsanity.com/paper/2308.02226/) (2023)
- [Standardize: Aligning Language Models with Expert-Defined Standards for Content Generation](https://mlsanity.com/paper/2402.12593/) (2024)
- [An Effective Automated Speaking Assessment Approach to Mitigating Data Scarcity and Imbalanced Distribution](https://mlsanity.com/paper/2404.07575/) (2024)
- [Meta-Tuning LLMs to Leverage Lexical Knowledge for Generalizable Language Style Understanding](https://mlsanity.com/paper/2305.14592/) (2023)
- [Trans-EnV: A Framework for Evaluating the Linguistic Robustness of LLMs Against English Varieties](https://mlsanity.com/paper/2505.20875/) (2025)
- [Exploiting the English Vocabulary Profile for L2 word-level vocabulary assessment with LLMs](https://mlsanity.com/paper/2506.02758/) (2025)
- [Ace-CEFR -- A Dataset for Automated Evaluation of the Linguistic Difficulty of Conversational Texts for LLM Applications](https://mlsanity.com/paper/2506.14046/) (2025)
- [Readability Reconsidered: A Cross-Dataset Analysis of Reference-Free Metrics](https://mlsanity.com/paper/2510.15345/) (2025)
- [Educational Cone Model in Embedding Vector Spaces](https://mlsanity.com/paper/2512.04227/) (2025)
- [This Treatment Works, Right? Evaluating LLM Sensitivity to Patient Question Framing in Medical QA](https://mlsanity.com/paper/2604.05051/) (2026)
- [EvalYaks: Instruction Tuning Datasets and LoRA Fine-tuned Models for Automated Scoring of CEFR B2 Speaking Assessment Transcripts](https://mlsanity.com/paper/2408.12226/) (2024)
- [Eye Tracking Based Cognitive Evaluation of Automatic Readability Assessment Methods](https://mlsanity.com/paper/2502.11150/) (2025)
- [Let's Simplify Step by Step: Guiding LLM Towards Multilingual Unsupervised Proficiency-Controlled Sentence Simplification](https://mlsanity.com/paper/2602.07499/) (2026)
- [An Effective Strategy for Modeling Score Ordinality and Non-uniform Intervals in Automated Speaking Assessment](https://mlsanity.com/paper/2509.03372/) (2025)
- [Towards Ontology-Based Descriptions of Conversations with Qualitatively-Defined Concepts](https://mlsanity.com/paper/2509.04926/) (2025)
- [Controlling Language Difficulty in Dialogues with Linguistic Features](https://mlsanity.com/paper/2509.14545/) (2025)
- [OUNLP at TSAR 2025 Shared Task: Multi-Round Text Simplifier via Code Generation](https://mlsanity.com/paper/2511.04495/) (2025)
- [Readability Measures and Automatic Text Simplification: In the Search of a Construct](https://mlsanity.com/paper/2511.09536/) (2025)
- [Textual Data Bias Detection and Mitigation -- An Extensible Pipeline with Experimental Evaluation](https://mlsanity.com/paper/2512.10734/) (2025)
- [Retcon -- a Prompt-Based Technique for Precise Control of LLMs in Conversations](https://mlsanity.com/paper/2603.03317/) (2026)
- [Predicting Disagreement with Human Raters in LLM-as-a-Judge Difficulty Assessment without Using Generation-Time Probability Signals](https://mlsanity.com/paper/2605.12422/) (2026)
- [ComplexityMT: Benchmarking the Interaction Between Text Complexity and Machine Translation](https://mlsanity.com/paper/2606.05421/) (2026)
- [Analyzing Traditional and Neural Approaches to Multilingual Readability Assessment](https://mlsanity.com/paper/2609.10792/) (2026)
- [Beetle: A Bilingual Model Suite for Modelling Second-Language Processing](https://mlsanity.com/paper/2609.22633/) (2026)
- [Evaluation of pre-trained models for pedagogical assessment of novel AI-assisted educational questions](https://mlsanity.com/paper/2609.27749/) (2026)

## Abstract (arXiv)

Controllable text simplification is a crucial assistive technique for language learning and teaching. One of the primary factors hindering its advancement is the lack of a corpus annotated with sentence difficulty levels based on language ability descriptions. To address this problem, we created the CEFR-based Sentence Profile (CEFR-SP) corpus, containing 17k English sentences annotated with the levels based on the Common European Framework of Reference for Languages assigned by English-education professionals. In addition, we propose a sentence-level assessment model to handle unbalanced level distribution because the most basic and highly proficient sentences are naturally scarce. In the experiments in this study, our method achieved a macro-F1 score of 84.5% in the level assessment, thus outperforming strong baselines employed in readability assessment.