# Text Simplification with Sentence Embeddings

arXiv: https://arxiv.org/abs/2510.24365 · HTML: https://mlsanity.com/paper/2510.24365/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 45 references that are in the map.
- First posted on arXiv 28 Oct 2025.

## Builds on (most cited first, 19 of 19)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [No Language Left Behind: Scaling Human-Centered Machine Translation](https://mlsanity.com/paper/2207.04672/) (2022)
- [BLEURT: Learning Robust Metrics for Text Generation](https://mlsanity.com/paper/2004.04696/) (2020)
- [Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference](https://mlsanity.com/paper/2412.13663/) (2024)
- [SONAR: Sentence-Level Multimodal and Language-Agnostic Representations](https://mlsanity.com/paper/2308.11466/) (2023)
- [Neural CRF Model for Sentence Alignment in Text Simplification](https://mlsanity.com/paper/2005.02324/) (2020)
- [Large Concept Models: Language Modeling in a Sentence Representation Space](https://mlsanity.com/paper/2412.08821/) (2024)
- [EditNTS: An Neural Programmer-Interpreter Model for Sentence Simplification through Explicit Editing](https://mlsanity.com/paper/1906.08104/) (2019)
- [LENS: A Learnable Evaluation Metric for Text Simplification](https://mlsanity.com/paper/2212.09739/) (2022)
- [BLESS: Benchmarking Large Language Models on Sentence Simplification](https://mlsanity.com/paper/2310.15773/) (2023)
- [Document-Level Text Simplification: Dataset, Criteria and Baseline](https://mlsanity.com/paper/2110.05071/) (2021)
- [Med-EASi: Finely Annotated Dataset and Models for Controllable Simplification of Medical Texts](https://mlsanity.com/paper/2302.09155/) (2023)
- [A Neural Pairwise Ranking Model for Readability Assessment](https://mlsanity.com/paper/2203.07450/) (2022)
- [DEPLAIN: A German Parallel Corpus with Intralingual Translations into Plain Language for Sentence and Document Simplification](https://mlsanity.com/paper/2305.18939/) (2023)
- [Benchmarking Automated Clinical Language Simplification: Dataset, Algorithm, and Evaluation](https://mlsanity.com/paper/2012.02420/) (2020)
- [SONAR-LLM: Autoregressive Transformer that Thinks in Sentence Embeddings and Speaks in Tokens](https://mlsanity.com/paper/2508.05305/) (2025)

## Abstract (arXiv)

Sentence embeddings can be decoded to give approximations of the original texts used to create them. We explore this effect in the context of text simplification, demonstrating that reconstructed text embeddings preserve complexity levels. We experiment with a small feed forward neural network to effectively learn a transformation between sentence embeddings representing high-complexity and low-complexity texts. We provide comparison to a Seq2Seq and LLM-based approach, showing encouraging results in our much smaller learning setting. Finally, we demonstrate the applicability of our transformation to an unseen simplification dataset (MedEASI), as well as datasets from languages outside the training data (ES,DE). We conclude that learning transformations in sentence embedding space is a promising direction for future research and has potential to unlock the ability to develop small, but powerful models for text simplification and other natural language generation tasks.