mlsanityData to 14 Oct 2026

Swiss-Judgment-Prediction: A Multilingual Legal Judgment Prediction Benchmark

arXiv 2110.00806 ↗First posted 2 Oct 2021cs.CLcited by 60

Not on the map. Why?

Abstract (arXiv)

In many jurisdictions, the excessive workload of courts leads to high delays. Suitable predictive AI models can assist legal professionals in their work, and thus enhance and speed up the process. So far, Legal Judgment Prediction (LJP) datasets have been released in English, French, and Chinese. We publicly release a multilingual (German, French, and Italian), diachronic (2000-2020) corpus of 85K cases from the Federal Supreme Court of Switzerland (FSCS). We evaluate state-of-the-art BERT-based methods including two variants of BERT that overcome the BERT input (text) length limitation (up to 512 tokens). Hierarchical BERT has the best performance (approx. 68-70% Macro-F1-Score in German and French). Furthermore, we study how several factors (canton of origin, year of publication, text length, legal area) affect performance. We release both the benchmark dataset and our code to accelerate future research and ensure reproducibility.

Key facts. Cited by 60 papers in the map (data through 14 Oct 2026). Builds on 26 of its 54 references that are in the map. First posted on arXiv 2 Oct 2021.

Builds on

26 of 54 references.

References · 26

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  3. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,899
  4. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,375
  5. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  6. 2020Longformer: The Long-Document Transformercited by 3,413
  7. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,811
  8. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,790
  9. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,989
  10. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,729
  11. 2020Big Bird: Transformers for Longer Sequencescited by 1,684
  12. 2020WILDS: A Benchmark of in-the-Wild Distribution Shiftscited by 1,375
14 more
  1. 2020MAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfercited by 445
  2. 2019CamemBERT: a Tasty French Language Modelcited by 368
  3. 2019Neural Legal Judgment Prediction in Englishcited by 209
  4. 2018CAIL2018: A Large-Scale Legal Dataset for Judgment Predictioncited by 170
  5. 2020How Does NLP Benefit Legal System: A Summary of Legal Artificial Intelligencecited by 129
  6. 2021Paragraph-level Rationale Extraction through Regularization: A case study on European Court of Human Rights Casescited by 77
  7. 2020We Need to Talk About Random Splitscited by 71
  8. 2017Learning to Predict Charges for Criminal Cases with Legal Basiscited by 61
  9. 2021Changing the World by Changing the Datacited by 51
  10. 2018Interpretable Charge Predictions for Criminal Cases: Learning to Generate Court Views from Fact Descriptionscited by 36
  11. 2017Predicting the Law Area and Decisions of French Supreme Court Casescited by 30
  12. 2021On the Ethical Limits of Natural Language Processing on Legal Textcited by 26
  13. 2018Automatic Judgment Prediction via Legal Reading Comprehensioncited by 19
  14. 2021Equality before the Law: Legal Judgment Consistency Analysis for Fairnesscited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 8; 2023: 12; 2024: 14; 2025: 16; 2026: 10.

60 papers.

Sort:

By area:

  1. 2024SaulLM-7B: A pioneering Large Language Model for Lawcited by 142
  2. 2023LEXTREME: A Multi-Lingual and Multi-Task Benchmark for the Legal Domaincited by 68
  3. 2023MultiLegalPile: A 689GB Multilingual Legal Corpuscited by 60
  4. 2022Legal Prompt Engineering for Multilingual Legal Judgement Predictioncited by 55
  5. 2023Precedent-Enhanced Legal Judgment Prediction with LLM and Domain-Model Collaborationcited by 52
  6. 2022A Survey on Legal Judgment Prediction: Datasets, Metrics, Models and Challengescited by 50
  7. 2022FairLex: A Multilingual Benchmark for Evaluating Fairness in Legal Text Processingcited by 49
  8. 2022A Multi-Task Benchmark for Korean Legal Language Understanding and Judgement Predictioncited by 44
  9. 2024Natural Language Processing for the Legal Domain: A Survey of Tasks, Datasets, Models, and Challengescited by 42
  10. 2022A Survey of Methods for Addressing Class Imbalance in Deep-Learning Based Natural Language Processingcited by 21
  11. 2023Multi-Defendant Legal Judgment Prediction via Hierarchical Reasoningcited by 19
  12. 2022ClassActionPrediction: A Challenging Benchmark for Legal Judgment Prediction of Class Action Cases in the UScited by 17
38 more
  1. 2022An Empirical Study on Cross-X Transfer for Legal Judgment Predictioncited by 16
  2. 2024Legal Judgment Reimagined: PredEx and the Rise of Intelligent AI Interpretation in Indian Courtscited by 15
  3. 2026PLawBench: A Rubric-Based Benchmark for Evaluating LLMs in Real-World Legal Practicecited by 15
  4. 2023Zero-shot Transfer of Article-aware Legal Outcome Classification for European Court of Human Rights Casescited by 14
  5. 2022BudgetLongformer: Can we Cheaply Pretrain a SotA Legal Language Model From Scratch?cited by 13
  6. 2023Bringing order into the realm of Transformer-based language models for artificial intelligence and lawcited by 13
  7. 2023Leveraging Task Dependency and Contrastive Learning for Case Outcome Classification on European Court of Human Rights Casescited by 12
  8. 2024ChronosLex: Time-aware Incremental Training for Temporal Generalization of Legal Classification Taskscited by 11
  9. 2025SwiLTra-Bench: The Swiss Legal Translation Benchmarkcited by 11
  10. 2024Rethinking Legal Judgement Prediction in a Realistic Scenario in the Era of Large Language Modelscited by 10
  11. 2025Large Language Models Meet Legal Artificial Intelligence: A Surveycited by 10
  12. 2023MultiLegalSBD: A Multilingual Legal Sentence Boundary Detection Datasetcited by 7
  13. 2024Towards Explainability and Fairness in Swiss Judgement Prediction: Benchmarking on a Multilingual Datasetcited by 7
  14. 2025NyayaRAG: Realistic Legal Judgment Prediction with RAG under the Indian Common Law Systemcited by 6
  15. 2024Legal Fact Prediction: The Missing Piece in Legal Judgment Predictioncited by 5
  16. 2024Towards Explainability in Legal Outcome Prediction Modelscited by 4
  17. 2024Incorporating Precedents for Legal Judgement Prediction on European Court of Human Rights Casescited by 4
  18. 2024Developing a Pragmatic Benchmark for Assessing Korean Legal Language Understanding in Large Language Modelscited by 4
  19. 2024NyayaAnumana & INLegalLlama: The Largest Indian Legal Judgment Prediction Dataset and Specialized Language Model for Enhanced Decision Analysiscited by 4
  20. 2023Resolving Legalese: A Multilingual Exploration of Negation Scope Resolution in Legal Documentscited by 3
  21. 2023Automatic Anonymization of Swiss Federal Supreme Court Rulingscited by 3
  22. 2025AnnoCaseLaw: A Richly-Annotated Dataset For Benchmarking Explainable Legal Judgment Predictioncited by 3
  23. 2025ASP2LJ : An Adversarial Self-Play Laywer Augmented Legal Judgment Frameworkcited by 3
  24. 2025When Large Language Models Meet Law: Dual-Lens Taxonomy, Technical Advances, and Ethical Governancecited by 3
  25. 2025Computational Law: Datasets, Benchmarks, and Ontologiescited by 2
  26. 2025Can LLMs Identify Tax Abuse?cited by 2
  27. 2026CanLegalRAGBench: Evaluating Retrieval-Augmented Generation on Canadian Case Lawcited by 2
  28. 2023Adversarially Robust Neural Legal Judgement Systemscited by 1
  29. 2024The Craft of Selective Prediction: Towards Reliable Case Outcome Classification -- An Empirical Study on European Court of Human Rights Casescited by 1
  30. 2024From Citations to Criticality: Predicting Legal Decision Influence in the Multilingual Swiss Jurisprudencecited by 1
  31. 2025LegalViz: Legal Text Visualization by Text To Diagram Generationcited by 1
  32. 2025TathyaNyaya and FactLegalLlama: Advancing Factual Judgment Prediction and Explanation in the Indian Legal Contextcited by 1
  33. 2025The Judge Variable: Challenging Judge-Agnostic Legal Judgment Predictioncited by 1
  34. 2026Evaluation of Large Language Models in Legal Applications: Challenges, Methods, and Future Directionscited by 1
  35. 2026Swiss-Bench SBP-002: A Frontier Model Comparison on Swiss Legal and Regulatory Taskscited by 1
  36. 2026Towards Explainable Adjudicative Variance: Quantifying Judicial Discretion via Gated Multi-Task Learningcited by 1
  37. 2023FAIR: A Causal Framework for Accurately Inferring Judgments Reversalscited by 0
  38. 2024Towards Supporting Legal Argumentation with NLP: Is More Data Really All You Need?cited by 0

50 most cited of 60 All 60 (JSON)