mlsanityData to 14 Oct 2026

Automating IRAC Analysis in Malaysian Contract Law using a Semi-Structured Knowledge Base

arXiv 2406.13217 ↗First posted 19 Jun 2024cs.CLcited by 5

Not on the map. Why?

Abstract (arXiv)

The effectiveness of Large Language Models (LLMs) in legal reasoning is often limited due to the unique legal terminologies and the necessity for highly specialized knowledge. These limitations highlight the need for high-quality data tailored for complex legal reasoning tasks. This paper introduces LegalSemi, a benchmark specifically curated for legal scenario analysis. LegalSemi comprises 54 legal scenarios, each rigorously annotated by legal experts, based on the comprehensive IRAC (Issue, Rule, Application, Conclusion) framework from Malaysian Contract Law. In addition, LegalSemi is accompanied by a structured knowledge base (SKE). A series of experiments were conducted to assess the usefulness of LegalSemi for IRAC analysis. The experimental results demonstrate the effectiveness of incorporating the SKE for issue identification, rule retrieval, application and conclusion generation using four different LLMs.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 24 of its 59 references that are in the map. First posted on arXiv 19 Jun 2024.

Builds on

24 of 59 references.

References · 24

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  4. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  5. 2023Reasoning with Language Model is Planning with World Modelcited by 861
  6. 2023Unifying Large Language Models and Knowledge Graphs: A Roadmapcited by 668
  7. 2020A Survey on Knowledge Graphs: Representation, Acquisition and Applicationscited by 643
  8. 2020Knowledge Graphscited by 621
  9. 2021CUAD: An Expert-Annotated NLP Dataset for Legal Contract Reviewcited by 236
  10. 2021When Does Pretraining Help? Assessing Self-Supervised Learning for Law and the CaseHOLD Datasetcited by 201
  11. 2019Question Answering for Privacy Policies: Combining Computational and Legal Perspectivescited by 67
  12. 2020A Dataset for Statutory Reasoning in Tax Law Entailment and Question Answeringcited by 62
12 more
  1. 2022Legal Prompting: Teaching a Language Model to Think Like a Lawyercited by 60
  2. 2023LeXFiles and LegalLAMA: Facilitating English Multinational Legal Language Model Developmentcited by 54
  3. 2024Can LLMs Produce Faithful Explanations For Fact-checking? Towards Faithful Explainable Fact-Checking via Multi-Agent Debatecited by 48
  4. 2023Legal Syllogism Prompting: Teaching Large Language Models for Legal Judgment Predictioncited by 38
  5. 2020A Benchmark for Lease Contract Reviewcited by 26
  6. 2021LeSICiN: A Heterogeneous Graph-based Approach for Automatic Legal Statute Identification from Indian Legal Documentscited by 23
  7. 2021Factoring Statutory Reasoning as Language Understanding Challengescited by 20
  8. 2021Reasoning Over Virtual Knowledge Bases With Open Predicate Relationscited by 18
  9. 2022LegalBench: Prototyping a Collaborative Benchmark for Legal Reasoningcited by 17
  10. 2023Can ChatGPT Perform Reasoning Using the IRAC Method in Analyzing Legal Scenarios Like a Lawyer?cited by 12
  11. 2023Finding the Law: Enhancing Statutory Article Retrieval via Graph Neural Networkscited by 9
  12. 2024CuSINeS: Curriculum-driven Structure Induced Negative Sampling for Statutory Article Retrievalcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 2.

5 papers.

  1. 2025Investigating the Shortcomings of LLMs in Step-by-Step Legal Reasoningcited by 9
  2. 2025G-reasoner: Foundation Models for Unified Reasoning over Graph-structured Knowledgecited by 9
  3. 2025Evaluating LLM-Generated Legal Explanations for Regulatory Compliance in Social Media Influencer Marketingcited by 4
  4. 2026LePREC: Reasoning as Classification over Structured Factors for Assessing Relevance of Legal Issuescited by 0
  5. 2026From Judgments to Issues: Structured Extraction of Legal Reasoning with Citation-Hallucination Controlcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023LegalBench: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Models
  2. 2026ContractScrub: A benchmark for final review of legal contracts
  3. 2026LAURA: Knowledge Distillation for Interpretable Ambiguous Clause Identification in Legal Contracts
  4. 2026Expert Evaluation of LLM's Open-Ended Legal Reasoning on the Japanese Bar Exam Writing Task
  5. 2023LawBench: Benchmarking Legal Knowledge of Large Language Models
  6. 2026Evaluation of Large Language Models in Legal Applications: Challenges, Methods, and Future Directions
  7. 2025Metadata Extraction Leveraging Large Language Models
  8. 2025LeMAJ (Legal LLM-as-a-Judge): Bridging Legal Reasoning and LLM Evaluation