# Predicting the Outcome of Judicial Decisions made by the European Court of Human Rights

arXiv: https://arxiv.org/abs/1912.10819 · HTML: https://mlsanity.com/paper/1912.10819/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Dec 2019.

## Built on by (most cited first, 4 of 4)

- [The Cambridge Law Corpus: A Dataset for Legal AI Research](https://mlsanity.com/paper/2309.12269/) (2023)
- [The CLC-UKET Dataset: Benchmarking Case Outcome Prediction for the UK Employment Tribunal](https://mlsanity.com/paper/2409.08098/) (2024)
- [A New Task and Dataset on Detecting Attacks on Human Rights Defenders](https://mlsanity.com/paper/2306.17695/) (2023)
- [Unsupervised Simplification of Legal Texts](https://mlsanity.com/paper/2209.00557/) (2022)

## Abstract (arXiv)

In this study, machine learning models were constructed to predict whether judgments made by the European Court of Human Rights (ECHR) would lead to a violation of an Article in the Convention on Human Rights. The problem is framed as a binary classification task where a judgment can lead to a "violation" or "non-violation" of a particular Article. Using auto-sklearn, an automated algorithm selection package, models were constructed for 12 Articles in the Convention. To train these models, textual features were obtained from the ECHR Judgment documents using N-grams, word embeddings and paragraph embeddings. Additional documents, from the ECHR, were incorporated into the models through the creation of a word embedding (echr2vec) and a doc2vec model. The features obtained using the echr2vec embedding provided the highest cross-validation accuracy for 5 of the Articles. The overall test accuracy, across the 12 Articles, was 68.83%. As far as we could tell, this is the first estimate of the accuracy of such machine learning models using a realistic test set. This provides an important benchmark for future work. As a baseline, a simple heuristic of always predicting the most common outcome in the past was used. The heuristic achieved an overall test accuracy of 86.68% which is 29.7% higher than the models. Again, this was seemingly the first study that included such a heuristic with which to compare model results. The higher accuracy achieved by the heuristic highlights the importance of including such a baseline.