# Improving Estonian Text Simplification through Pretrained Language Models and Custom Datasets

arXiv: https://arxiv.org/abs/2501.15624 · HTML: https://mlsanity.com/paper/2501.15624/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 25 references that are in the map.
- First posted on arXiv 26 Jan 2025.

## Builds on (most cited first, 8 of 8)

- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Sentence Simplification with Deep Reinforcement Learning](https://mlsanity.com/paper/1703.10931/) (2017)
- [Neural CRF Model for Sentence Alignment in Text Simplification](https://mlsanity.com/paper/2005.02324/) (2020)
- [Controllable Sentence Simplification](https://mlsanity.com/paper/1910.02677/) (2019)
- [When "A Helpful Assistant" Is Not Really Helpful: Personas in System Prompts Do Not Improve Performances of Large Language Models](https://mlsanity.com/paper/2311.10054/) (2023)
- [Integrating Transformer and Paraphrase Rules for Sentence Simplification](https://mlsanity.com/paper/1810.11193/) (2018)
- [Analysing Zero-Shot Readability-Controlled Sentence Simplification](https://mlsanity.com/paper/2409.20246/) (2024)
- [Unsupervised Sentence Simplification via Dependency Parsing](https://mlsanity.com/paper/2206.12261/) (2022)

## Built on by (most cited first, 1 of 1)

- [Cross-Lingual Stability of LLM Judges Under Controlled Generation: Evidence from Finno-Ugric Languages](https://mlsanity.com/paper/2602.02287/) (2026)

## Abstract (arXiv)

This paper presents a method for text simplification based on two neural architectures: a neural machine translation (NMT) model and a fine-tuned large language model (LLaMA). Given the scarcity of existing resources for Estonian, a new dataset was created by combining manually translated corpora with GPT-4.0-generated simplifications. OpenNMT was selected as a representative NMT-based system, while LLaMA was fine-tuned on the constructed dataset. Evaluation shows LLaMA outperforms OpenNMT in grammaticality, readability, and meaning preservation. These results underscore the effectiveness of large language models for text simplification in low-resource language settings. The complete dataset, fine-tuning scripts, and evaluation pipeline are provided in a publicly accessible supplementary package to support reproducibility and adaptation to other languages.