# Ace-CEFR -- A Dataset for Automated Evaluation of the Linguistic Difficulty of Conversational Texts for LLM Applications

arXiv: https://arxiv.org/abs/2506.14046 · HTML: https://mlsanity.com/paper/2506.14046/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 49 references that are in the map.
- First posted on arXiv 16 Jun 2025.

## Builds on (most cited first, 8 of 8)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Neural CRF Model for Sentence Alignment in Text Simplification](https://mlsanity.com/paper/2005.02324/) (2020)
- [Supervised and Unsupervised Neural Approaches to Text Readability](https://mlsanity.com/paper/1907.11779/) (2019)
- [CEFR-Based Sentence Difficulty Annotation and Assessment](https://mlsanity.com/paper/2210.11766/) (2022)
- [BERT Embeddings for Automatic Readability Assessment](https://mlsanity.com/paper/2106.07935/) (2021)
- [ReadNet: A Hierarchical Transformer Framework for Web Article Readability Analysis](https://mlsanity.com/paper/2103.04083/) (2021)
- [Are pre-trained text representations useful for multilingual and multi-dimensional language proficiency modeling?](https://mlsanity.com/paper/2102.12971/) (2021)
- [Predicting CEFRL levels in learner English on the basis of metrics and full texts](https://mlsanity.com/paper/1806.11099/) (2018)

## Built on by (most cited first, 4 of 4)

- [Glite ARF: Verifier-Driven Research with Parallel LLM Coding Agents](https://mlsanity.com/paper/2606.27416/) (2026)
- [Retcon -- a Prompt-Based Technique for Precise Control of LLMs in Conversations](https://mlsanity.com/paper/2603.03317/) (2026)
- [Right at My Level: A Unified Multilingual Framework for Proficiency-Aware Text Simplification](https://mlsanity.com/paper/2604.05302/) (2026)
- [Towards Reducing Foreign Language Anxiety Using Level-Appropriate Embodied Conversational Agents](https://mlsanity.com/paper/2607.21887/) (2026)

## Abstract (arXiv)

There is an unmet need to evaluate the language difficulty of short, conversational passages of text, particularly for training and filtering Large Language Models (LLMs). We introduce Ace-CEFR, a dataset of English conversational text passages expert-annotated with their corresponding level of text difficulty. We experiment with several models on Ace-CEFR, including Transformer-based models and LLMs. We show that models trained on Ace-CEFR can measure text difficulty more accurately than human experts and have latency appropriate to production environments. Finally, we release the Ace-CEFR dataset to the public for research and development.