# Swiss-Judgment-Prediction: A Multilingual Legal Judgment Prediction Benchmark

arXiv: https://arxiv.org/abs/2110.00806 · HTML: https://mlsanity.com/paper/2110.00806/

## Key facts

- Cited by 60 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 54 references that are in the map.
- First posted on arXiv 2 Oct 2021.

## Builds on (most cited first, 26 of 26)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Parameter-Efficient Transfer Learning for NLP](https://mlsanity.com/paper/1902.00751/) (2019)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing](https://mlsanity.com/paper/1808.06226/) (2018)
- [SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems](https://mlsanity.com/paper/1905.00537/) (2019)
- [Big Bird: Transformers for Longer Sequences](https://mlsanity.com/paper/2007.14062/) (2020)
- [WILDS: A Benchmark of in-the-Wild Distribution Shifts](https://mlsanity.com/paper/2012.07421/) (2020)
- [MAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfer](https://mlsanity.com/paper/2005.00052/) (2020)
- [CamemBERT: a Tasty French Language Model](https://mlsanity.com/paper/1911.03894/) (2019)
- [Neural Legal Judgment Prediction in English](https://mlsanity.com/paper/1906.02059/) (2019)
- [CAIL2018: A Large-Scale Legal Dataset for Judgment Prediction](https://mlsanity.com/paper/1807.02478/) (2018)
- [How Does NLP Benefit Legal System: A Summary of Legal Artificial Intelligence](https://mlsanity.com/paper/2004.12158/) (2020)
- [Paragraph-level Rationale Extraction through Regularization: A case study on European Court of Human Rights Cases](https://mlsanity.com/paper/2103.13084/) (2021)
- [We Need to Talk About Random Splits](https://mlsanity.com/paper/2005.00636/) (2020)
- [Learning to Predict Charges for Criminal Cases with Legal Basis](https://mlsanity.com/paper/1707.09168/) (2017)
- [Changing the World by Changing the Data](https://mlsanity.com/paper/2105.13947/) (2021)
- [Interpretable Charge Predictions for Criminal Cases: Learning to Generate Court Views from Fact Descriptions](https://mlsanity.com/paper/1802.08504/) (2018)
- [Predicting the Law Area and Decisions of French Supreme Court Cases](https://mlsanity.com/paper/1708.01681/) (2017)
- [On the Ethical Limits of Natural Language Processing on Legal Text](https://mlsanity.com/paper/2105.02751/) (2021)
- [Automatic Judgment Prediction via Legal Reading Comprehension](https://mlsanity.com/paper/1809.06537/) (2018)
- [Equality before the Law: Legal Judgment Consistency Analysis for Fairness](https://mlsanity.com/paper/2103.13868/) (2021)

## Built on by (most cited first, 30 of 60)

- [SaulLM-7B: A pioneering Large Language Model for Law](https://mlsanity.com/paper/2403.03883/) (2024)
- [LEXTREME: A Multi-Lingual and Multi-Task Benchmark for the Legal Domain](https://mlsanity.com/paper/2301.13126/) (2023)
- [MultiLegalPile: A 689GB Multilingual Legal Corpus](https://mlsanity.com/paper/2306.02069/) (2023)
- [Legal Prompt Engineering for Multilingual Legal Judgement Prediction](https://mlsanity.com/paper/2212.02199/) (2022)
- [Precedent-Enhanced Legal Judgment Prediction with LLM and Domain-Model Collaboration](https://mlsanity.com/paper/2310.09241/) (2023)
- [A Survey on Legal Judgment Prediction: Datasets, Metrics, Models and Challenges](https://mlsanity.com/paper/2204.04859/) (2022)
- [FairLex: A Multilingual Benchmark for Evaluating Fairness in Legal Text Processing](https://mlsanity.com/paper/2203.07228/) (2022)
- [A Multi-Task Benchmark for Korean Legal Language Understanding and Judgement Prediction](https://mlsanity.com/paper/2206.05224/) (2022)
- [Natural Language Processing for the Legal Domain: A Survey of Tasks, Datasets, Models, and Challenges](https://mlsanity.com/paper/2410.21306/) (2024)
- [A Survey of Methods for Addressing Class Imbalance in Deep-Learning Based Natural Language Processing](https://mlsanity.com/paper/2210.04675/) (2022)
- [Multi-Defendant Legal Judgment Prediction via Hierarchical Reasoning](https://mlsanity.com/paper/2312.05762/) (2023)
- [ClassActionPrediction: A Challenging Benchmark for Legal Judgment Prediction of Class Action Cases in the US](https://mlsanity.com/paper/2211.00582/) (2022)
- [An Empirical Study on Cross-X Transfer for Legal Judgment Prediction](https://mlsanity.com/paper/2209.12325/) (2022)
- [Legal Judgment Reimagined: PredEx and the Rise of Intelligent AI Interpretation in Indian Courts](https://mlsanity.com/paper/2406.04136/) (2024)
- [PLawBench: A Rubric-Based Benchmark for Evaluating LLMs in Real-World Legal Practice](https://mlsanity.com/paper/2601.16669/) (2026)
- [Zero-shot Transfer of Article-aware Legal Outcome Classification for European Court of Human Rights Cases](https://mlsanity.com/paper/2302.00609/) (2023)
- [BudgetLongformer: Can we Cheaply Pretrain a SotA Legal Language Model From Scratch?](https://mlsanity.com/paper/2211.17135/) (2022)
- [Bringing order into the realm of Transformer-based language models for artificial intelligence and law](https://mlsanity.com/paper/2308.05502/) (2023)
- [Leveraging Task Dependency and Contrastive Learning for Case Outcome Classification on European Court of Human Rights Cases](https://mlsanity.com/paper/2302.00768/) (2023)
- [ChronosLex: Time-aware Incremental Training for Temporal Generalization of Legal Classification Tasks](https://mlsanity.com/paper/2405.14211/) (2024)
- [SwiLTra-Bench: The Swiss Legal Translation Benchmark](https://mlsanity.com/paper/2503.01372/) (2025)
- [Rethinking Legal Judgement Prediction in a Realistic Scenario in the Era of Large Language Models](https://mlsanity.com/paper/2410.10542/) (2024)
- [Large Language Models Meet Legal Artificial Intelligence: A Survey](https://mlsanity.com/paper/2509.09969/) (2025)
- [MultiLegalSBD: A Multilingual Legal Sentence Boundary Detection Dataset](https://mlsanity.com/paper/2305.01211/) (2023)
- [Towards Explainability and Fairness in Swiss Judgement Prediction: Benchmarking on a Multilingual Dataset](https://mlsanity.com/paper/2402.17013/) (2024)
- [NyayaRAG: Realistic Legal Judgment Prediction with RAG under the Indian Common Law System](https://mlsanity.com/paper/2508.00709/) (2025)
- [Legal Fact Prediction: The Missing Piece in Legal Judgment Prediction](https://mlsanity.com/paper/2409.07055/) (2024)
- [Towards Explainability in Legal Outcome Prediction Models](https://mlsanity.com/paper/2403.16852/) (2024)
- [Incorporating Precedents for Legal Judgement Prediction on European Court of Human Rights Cases](https://mlsanity.com/paper/2409.18644/) (2024)
- [Developing a Pragmatic Benchmark for Assessing Korean Legal Language Understanding in Large Language Models](https://mlsanity.com/paper/2410.08731/) (2024)

## Abstract (arXiv)

In many jurisdictions, the excessive workload of courts leads to high delays. Suitable predictive AI models can assist legal professionals in their work, and thus enhance and speed up the process. So far, Legal Judgment Prediction (LJP) datasets have been released in English, French, and Chinese. We publicly release a multilingual (German, French, and Italian), diachronic (2000-2020) corpus of 85K cases from the Federal Supreme Court of Switzerland (FSCS). We evaluate state-of-the-art BERT-based methods including two variants of BERT that overcome the BERT input (text) length limitation (up to 512 tokens). Hierarchical BERT has the best performance (approx. 68-70% Macro-F1-Score in German and French). Furthermore, we study how several factors (canton of origin, year of publication, text length, legal area) affect performance. We release both the benchmark dataset and our code to accelerate future research and ensure reproducibility.