mlsanityData to 14 Oct 2026

How Much is a Human Right Worth? ECtHR-NPD: A Benchmark for Predicting Non-Pecuniary Damage Awards

arXiv 2609.18908 ↗First posted 16 Sep 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Existing legal benchmarks cover diverse tasks, while continuous monetary remedies remain comparatively underexplored. We introduce ECtHR-NPD, to the best of our knowledge, the first benchmark for predicting non-pecuniary damage (NPD) awards at the European Court of Human Rights (ECtHR) from case information when no statutory formula or explicit calculation rule determines the amount. ECtHR-NPD contains 14,575 cases with case-level awards in nominal euros, chronological splits, and a protocol separating target construction from model input. We evaluate a battery of methods, including constant predictors, gradient-boosted trees, retrieval methods, fine-tuned encoder language models (LMs), prompted decoder LMs, and knowledge-augmented agents. Our results show that more sophisticated LM and agentic approaches do not consistently outperform the strongest feature-based baseline. All model families struggle to identify zero awards and to calibrate high-award predictions, with further degradation on the Challenging test view, making ECtHR-NPD a challenging testbed for current state-of-the-art open-weight and proprietary LMs.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 32 of its 75 references that are in the map. First posted on arXiv 16 Sep 2026.

Builds on

32 of 75 references.

References · 32

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  2. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,166
  3. 2025gpt-oss-120b & gpt-oss-20b Model Cardcited by 1,386
  4. 2024M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillationcited by 1,073
  5. 2024Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inferencecited by 554
  6. 2021LexGLUE: A Benchmark Dataset for Legal Language Understanding in Englishcited by 310
  7. 2021CUAD: An Expert-Annotated NLP Dataset for Legal Contract Reviewcited by 236
  8. 2019Neural Legal Judgment Prediction in Englishcited by 209
  9. 2021When Does Pretraining Help? Assessing Self-Supervised Learning for Law and the CaseHOLD Datasetcited by 201
  10. 2018CAIL2018: A Large-Scale Legal Dataset for Judgment Predictioncited by 170
  11. 2019JEC-QA: A Legal-Domain Question Answering Datasetcited by 116
  12. 2022Pile of Law: Learning Responsible Data Filtering from the Law and a 256GB Open-Source Legal Datasetcited by 113
20 more
  1. 2024BM25S: Orders of magnitude faster lexical search via eager sparse scoringcited by 88
  2. 2021ILDC for CJPE: Indian Legal Documents Corpus for Court Judgment Prediction and Explanationcited by 78
  3. 2021Paragraph-level Rationale Extraction through Regularization: A case study on European Court of Human Rights Casescited by 77
  4. 2023LEXTREME: A Multi-Lingual and Multi-Task Benchmark for the Legal Domaincited by 68
  5. 2020A Dataset for Statutory Reasoning in Tax Law Entailment and Question Answeringcited by 62
  6. 2021Swiss-Judgment-Prediction: A Multilingual Legal Judgment Prediction Benchmarkcited by 60
  7. 2023MultiLegalPile: A 689GB Multilingual Legal Corpuscited by 60
  8. 2023LeXFiles and LegalLAMA: Facilitating English Multinational Legal Language Model Developmentcited by 54
  9. 2025LEXam: Benchmarking Legal Reasoning on 340 Law Examscited by 54
  10. 2019Legal Judgment Prediction via Multi-Perspective Bi-Feedback Networkcited by 50
  11. 2018Numeracy for Language Models: Evaluating and Improving their Ability to Predict Numberscited by 42
  12. 2022LEVEN: A Large-Scale Chinese Legal Event Detection Datasetcited by 36
  13. 2024IL-TUR: Benchmark for Indian Legal Text Understanding and Reasoningcited by 16
  14. 2024Legal Judgment Reimagined: PredEx and the Rise of Intelligent AI Interpretation in Indian Courtscited by 15
  15. 2023The Cambridge Law Corpus: A Dataset for Legal AI Researchcited by 14
  16. 2025Quantile Regression with Large Language Models for Price Predictioncited by 10
  17. 2020Deep Hurdle Networks for Zero-Inflated Multi-Target Regression: Application to Multiple Species Abundance Estimationcited by 9
  18. 2024Through the Lens of Split Vote: Exploring Disagreement, Difficulty and Calibration in Legal Case Outcome Classificationcited by 9
  19. 2022Exploiting Contrastive Learning and Numerical Evidence for Confusing Legal Judgment Predictioncited by 8
  20. 2024The Craft of Selective Prediction: Towards Reliable Case Outcome Classification -- An Empirical Study on European Court of Human Rights Casescited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Legal Reasoning Is Not Lawyering: Rethinking Legal Benchmarks for Pro Se Access to Justice
  2. 2026LegalRewardBench: Benchmarking Reward Models for Legal Preference Evaluation
  3. 2024LAR-ECHR: A New Legal Argument Reasoning Task and Dataset for Cases of the European Court of Human Rights
  4. 2026Can LLMs Reason in a Legally Meaningful Manner? A Small-scale Study on European Court of Human Rights Cases
  5. 2026Multi-Legal-Bench: Evaluating LLMs on Legal Reasoning Across Jurisdictions, Languages, and Legal Traditions
  6. 2026IntLawNER: A Named Entity Recognition Dataset and Benchmark in International Law
  7. 2025LeMAJ (Legal LLM-as-a-Judge): Bridging Legal Reasoning and LLM Evaluation
  8. 2026Building Legal Reward Models for Grounding and Abstention