NLP at UC Santa Cruz at SemEval-2024 Task 5: Legal Answer Validation using Few-Shot Multi-Choice QA
Not on the map. Why?
Abstract (arXiv)
This paper presents our submission to the SemEval 2024 Task 5: The Legal Argument Reasoning Task in Civil Procedure. We present two approaches to solving the task of legal answer validation, given an introduction to the case, a question and an answer candidate. Firstly, we fine-tuned pre-trained BERT-based models and found that models trained on domain knowledge perform better. Secondly, we performed few-shot prompting on GPT models and found that reformulating the answer validation task to be a multiple-choice QA task remarkably improves the performance of the model. Our best submission is a BERT-based model that achieved the 7th place out of 20.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 6 of its 12 references that are in the map. First posted on arXiv 4 Apr 2024.
Builds on
6 of 12 references.
References · 6
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2020Longformer: The Long-Document Transformercited by 3,413
- 2023A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPTcited by 619
- 2021When Does Pretraining Help? Assessing Self-Supervised Learning for Law and the CaseHOLD Datasetcited by 201
- 2023LeXFiles and LegalLAMA: Facilitating English Multinational Legal Language Model Developmentcited by 54
Built on by
Citing papers per year: 2026: 2.
2 papers.