# JUDGEBERT: Assessing Legal Meaning Preservation Between Sentences

arXiv: https://arxiv.org/abs/2508.16870 · HTML: https://mlsanity.com/paper/2508.16870/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 112 references that are in the map.
- First posted on arXiv 23 Aug 2025.

## Builds on (most cited first, 22 of 22)

- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Ethical and social risks of harm from Language Models](https://mlsanity.com/paper/2112.04359/) (2021)
- [On the Stability of Fine-tuning BERT: Misconceptions, Explanations, and Strong Baselines](https://mlsanity.com/paper/2006.04884/) (2020)
- [CUAD: An Expert-Annotated NLP Dataset for Legal Contract Review](https://mlsanity.com/paper/2103.06268/) (2021)
- [Survey of the State of the Art in Natural Language Generation: Core tasks, applications and evaluation](https://mlsanity.com/paper/1703.09902/) (2017)
- [With Little Power Comes Great Responsibility](https://mlsanity.com/paper/2010.06595/) (2020)
- [ASSET: A Dataset for Tuning and Evaluation of Sentence Simplification Models with Multiple Rewriting Transformations](https://mlsanity.com/paper/2005.00481/) (2020)
- [BLEU is Not Suitable for the Evaluation of Text Simplification](https://mlsanity.com/paper/1810.05995/) (2018)
- [LENS: A Learnable Evaluation Metric for Text Simplification](https://mlsanity.com/paper/2212.09739/) (2022)
- [EUR-Lex-Sum: A Multi- and Cross-lingual Dataset for Long-form Summarization in the Legal Domain](https://mlsanity.com/paper/2210.13448/) (2022)
- [Natural Language Processing in the Legal Domain](https://mlsanity.com/paper/2302.12039/) (2023)
- [BLESS: Benchmarking Large Language Models on Sentence Simplification](https://mlsanity.com/paper/2310.15773/) (2023)
- [The Summary Loop: Learning to Write Abstractive Summaries Without Examples](https://mlsanity.com/paper/2105.05361/) (2021)
- [Revisiting non-English Text Simplification: A Unified Multilingual Benchmark](https://mlsanity.com/paper/2305.15678/) (2023)
- [Data-QuestEval: A Referenceless Metric for Data-to-Text Semantic Evaluation](https://mlsanity.com/paper/2104.07555/) (2021)
- [The Human Evaluation Datasheet 1.0: A Template for Recording Details of Human Evaluation Experiments in NLP](https://mlsanity.com/paper/2103.09710/) (2021)
- [From Text to Structure: Using Large Language Models to Support the Development of Legal Expert Systems](https://mlsanity.com/paper/2311.04911/) (2023)
- [Do Text Simplification Systems Preserve Meaning? A Human Evaluation via Reading Comprehension](https://mlsanity.com/paper/2312.10126/) (2023)
- [JuriBERT: A Masked-Language Model Adaptation for French Legal Text](https://mlsanity.com/paper/2110.01485/) (2021)
- [Promises and pitfalls of artificial intelligence for legal applications](https://mlsanity.com/paper/2402.01656/) (2024)
- [CamemBERT 2.0: A Smarter French Language Model Aged to Perfection](https://mlsanity.com/paper/2411.08868/) (2024)

## Abstract (arXiv)

Simplifying text while preserving its meaning is a complex yet essential task, especially in sensitive domain applications like legal texts. When applied to a specialized field, like the legal domain, preservation differs significantly from its role in regular texts. This paper introduces FrJUDGE, a new dataset to assess legal meaning preservation between two legal texts. It also introduces JUDGEBERT, a novel evaluation metric designed to assess legal meaning preservation in French legal text simplification. JUDGEBERT demonstrates a superior correlation with human judgment compared to existing metrics. It also passes two crucial sanity checks, while other metrics did not: For two identical sentences, it always returns a score of 100%; on the other hand, it returns 0% for two unrelated sentences. Our findings highlight its potential to transform legal NLP applications, ensuring accuracy and accessibility for text simplification for legal practitioners and lay users.