# Profiling German Text Simplification with Interpretable Model-Fingerprints

arXiv: https://arxiv.org/abs/2601.13050 · HTML: https://mlsanity.com/paper/2601.13050/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 34 references that are in the map.
- First posted on arXiv 19 Jan 2026.

## Builds on (most cited first, 22 of 22)

- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [A Survey on LLM-as-a-Judge](https://mlsanity.com/paper/2411.15594/) (2024)
- [Breaking NLI Systems with Sentences that Require Simple Lexical Inferences](https://mlsanity.com/paper/1805.02266/) (2018)
- [ASSET: A Dataset for Tuning and Evaluation of Sentence Simplification Models with Multiple Rewriting Transformations](https://mlsanity.com/paper/2005.00481/) (2020)
- [BLEU is Not Suitable for the Evaluation of Text Simplification](https://mlsanity.com/paper/1810.05995/) (2018)
- [Controllable Sentence Simplification](https://mlsanity.com/paper/1910.02677/) (2019)
- [LENS: A Learnable Evaluation Metric for Text Simplification](https://mlsanity.com/paper/2212.09739/) (2022)
- [EASSE: Easier Automatic Sentence Simplification Evaluation](https://mlsanity.com/paper/1908.04567/) (2019)
- [Semantic Structural Evaluation for Text Simplification](https://mlsanity.com/paper/1810.05022/) (2018)
- [Revisiting non-English Text Simplification: A Unified Multilingual Benchmark](https://mlsanity.com/paper/2305.15678/) (2023)
- [Subjective Assessment of Text Complexity: A Dataset for German Language](https://mlsanity.com/paper/1904.07733/) (2019)
- [Language Models for German Text Simplification: Overcoming Parallel Data Scarcity through Style-specific Pre-training](https://mlsanity.com/paper/2305.12908/) (2023)
- [A New Aligned Simple German Corpus](https://mlsanity.com/paper/2209.01106/) (2022)
- [Which Shortcut Solution Do Question Answering Models Prefer to Learn?](https://mlsanity.com/paper/2211.16220/) (2022)
- [Redefining Simplicity: Benchmarking Large Language Models from Lexical to Document Simplification](https://mlsanity.com/paper/2502.08281/) (2025)
- [Which Spurious Correlations Impact Reasoning in NLI Models? A Visual Interactive Diagnosis through Data-Constrained Counterfactuals](https://mlsanity.com/paper/2306.12146/) (2023)
- [German Text Simplification: Finetuning Large Language Models with Semi-Synthetic Data](https://mlsanity.com/paper/2402.10675/) (2024)
- [An In-depth Evaluation of Large Language Models in Sentence Simplification with Error-based Human Assessment](https://mlsanity.com/paper/2403.04963/) (2024)
- [Evaluation Under Imperfect Benchmarks and Ratings: A Case Study in Text Simplification](https://mlsanity.com/paper/2504.09394/) (2025)
- [Digital Comprehensibility Assessment of Simplified Texts among Persons with Intellectual Disabilities](https://mlsanity.com/paper/2402.13094/) (2024)
- [REFeREE: A REference-FREE Model-Based Metric for Text Simplification](https://mlsanity.com/paper/2403.17640/) (2024)
- [Explaining Relation Classification Models with Semantic Extents](https://mlsanity.com/paper/2308.02193/) (2023)

## Abstract (arXiv)

While Large Language Models (LLMs) produce highly nuanced text simplifications, developers currently lack tools for a holistic, efficient, and reproducible diagnosis of their behavior. This paper introduces the Simplification Profiler, a diagnostic toolkit that generates a multidimensional, interpretable fingerprint of simplified texts. Multiple aggregated simplifications of a model result in a model's fingerprint. This novel evaluation paradigm is particularly vital for languages, where the data scarcity problem is magnified when creating flexible models for diverse target groups rather than a single, fixed simplification style. We propose that measuring a model's unique behavioral signature is more relevant in this context as an alternative to correlating metrics with human preferences. We operationalize this with a practical meta-evaluation of our fingerprints' descriptive power, which bypasses the need for large, human-rated datasets. This test measures if a simple linear classifier can reliably identify various model configurations by their created simplifications, confirming that our metrics are sensitive to a model's specific characteristics. The Profiler can distinguish high-level behavioral variations between prompting strategies and fine-grained changes from prompt engineering, including few-shot examples. Our complete feature set achieves classification F1-scores up to 71.9 %, improving upon simple baselines by over 48 percentage points. The Simplification Profiler thus offers developers a granular, actionable analysis to build more effective and truly adaptive text simplification systems.