mlsanityData to 7 Oct 2026

German Text Simplification: Finetuning Large Language Models with Semi-Synthetic Data

arXiv 2402.10675 ↗First posted 16 Feb 2024cs.CLcited by 4

On the map: Synthetic data and self-improvement for LLMs → LLM synthetic data generation

Abstract (arXiv)

This study pioneers the use of synthetically generated data for training generative models in document-level text simplification of German texts. We demonstrate the effectiveness of our approach with real-world online texts. Addressing the challenge of data scarcity in language simplification, we crawled professionally simplified German texts and synthesized a corpus using GPT-4. We finetune Large Language Models with up to 13 billion parameters on this data and evaluate their performance. This paper employs various methodologies for evaluation and demonstrates the limitations of currently used rule-based metrics. Both automatic and manual evaluations reveal that our models can significantly simplify real-world online texts, indicating the potential of synthetic data in improving text simplification.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in LLM synthetic data generation, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 8 of its 25 references that are in the map. First posted on arXiv 16 Feb 2024.

Builds on

8 of 25 references.

References · 8

  1. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  2. 2021WECHSEL: Effective initialization of subword embeddings for cross-lingual transfer of monolingual language modelscited by 89
  3. 2018BLEU is Not Suitable for the Evaluation of Text Simplificationcited by 78
  4. 2022Contrastive Search Is What You Need For Neural Text Generationcited by 58
  5. 2022LENS: A Learnable Evaluation Metric for Text Simplificationcited by 54
  6. 2023Language Models for German Text Simplification: Overcoming Parallel Data Scarcity through Style-specific Pre-trainingcited by 15
  7. 2022A New Aligned Simple German Corpuscited by 10
  8. 2023Using ChatGPT as a CAT tool in Easy Language translationcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 1.

4 papers.

  1. 2025German4All -- A Dataset and Model for Readability-Controlled Paraphrasing in Germancited by 2
  2. 2025A Typology of Synthetic Datasets for Dialogue Processing in Clinical Contextscited by 1
  3. 2025Evaluating the Effectiveness of Direct Preference Optimization for Personalizing German Automatic Text Simplifications for Persons with Intellectual Disabilitiescited by 0
  4. 2026Profiling German Text Simplification with Interpretable Model-Fingerprintscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Synthetic Data Generation with Large Language Models for Text Classification: Potential and Limitations
  2. 2025Scaling Laws of Synthetic Data for Language Models
  3. 2026How Can We Synthesize High-Quality Pretraining Data? A Systematic Study of Prompt Design, Generator Model, and Source Data
  4. 2025Aleph-Alpha-GermanWeb: Improving German-language LLM pre-training with model-based data curation and synthetic data generation
  5. 2025Understanding the Influence of Synthetic Data for Text Embedders
  6. 2023Generating Faithful Synthetic Data with Large Language Models: A Case Study in Computational Social Science
  7. 2025Demystifying Synthetic Data in LLM Pre-training: A Systematic Study of Scaling Laws, Benefits, and Pitfalls
  8. 2023Synthetic Data Generation in Low-Resource Settings via Fine-Tuning of Large Language Models