mlsanityData to 14 Oct 2026

JUDGEBERT: Assessing Legal Meaning Preservation Between Sentences

arXiv 2508.16870 ↗First posted 23 Aug 2025cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Simplifying text while preserving its meaning is a complex yet essential task, especially in sensitive domain applications like legal texts. When applied to a specialized field, like the legal domain, preservation differs significantly from its role in regular texts. This paper introduces FrJUDGE, a new dataset to assess legal meaning preservation between two legal texts. It also introduces JUDGEBERT, a novel evaluation metric designed to assess legal meaning preservation in French legal text simplification. JUDGEBERT demonstrates a superior correlation with human judgment compared to existing metrics. It also passes two crucial sanity checks, while other metrics did not: For two identical sentences, it always returns a score of 100%; on the other hand, it returns 0% for two unrelated sentences. Our findings highlight its potential to transform legal NLP applications, ensuring accuracy and accessibility for text simplification for legal practitioners and lay users.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 22 of its 112 references that are in the map. First posted on arXiv 23 Aug 2025.

Builds on

22 of 112 references.

References · 22

  1. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  2. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  3. 2021Ethical and social risks of harm from Language Modelscited by 966
  4. 2020On the Stability of Fine-tuning BERT: Misconceptions, Explanations, and Strong Baselinescited by 255
  5. 2021CUAD: An Expert-Annotated NLP Dataset for Legal Contract Reviewcited by 236
  6. 2017Survey of the State of the Art in Natural Language Generation: Core tasks, applications and evaluationcited by 206
  7. 2020With Little Power Comes Great Responsibilitycited by 142
  8. 2020ASSET: A Dataset for Tuning and Evaluation of Sentence Simplification Models with Multiple Rewriting Transformationscited by 101
  9. 2018BLEU is Not Suitable for the Evaluation of Text Simplificationcited by 78
  10. 2022LENS: A Learnable Evaluation Metric for Text Simplificationcited by 54
  11. 2022EUR-Lex-Sum: A Multi- and Cross-lingual Dataset for Long-form Summarization in the Legal Domaincited by 40
  12. 2023Natural Language Processing in the Legal Domaincited by 38
10 more
  1. 2023BLESS: Benchmarking Large Language Models on Sentence Simplificationcited by 35
  2. 2021The Summary Loop: Learning to Write Abstractive Summaries Without Examplescited by 27
  3. 2023Revisiting non-English Text Simplification: A Unified Multilingual Benchmarkcited by 27
  4. 2021Data-QuestEval: A Referenceless Metric for Data-to-Text Semantic Evaluationcited by 21
  5. 2021The Human Evaluation Datasheet 1.0: A Template for Recording Details of Human Evaluation Experiments in NLPcited by 19
  6. 2023From Text to Structure: Using Large Language Models to Support the Development of Legal Expert Systemscited by 17
  7. 2023Do Text Simplification Systems Preserve Meaning? A Human Evaluation via Reading Comprehensioncited by 16
  8. 2021JuriBERT: A Masked-Language Model Adaptation for French Legal Textcited by 14
  9. 2024Promises and pitfalls of artificial intelligence for legal applicationscited by 14
  10. 2024CamemBERT 2.0: A Smarter French Language Model Aged to Perfectioncited by 14

Built on by

Not cited yet.