# From Text to Structure: Using Large Language Models to Support the Development of Legal Expert Systems

arXiv: https://arxiv.org/abs/2311.04911 · HTML: https://mlsanity.com/paper/2311.04911/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 33 references that are in the map.
- First posted on arXiv 1 Nov 2023.

## Builds on (most cited first, 10 of 10)

- [Can GPT-3 Perform Statutory Reasoning?](https://mlsanity.com/paper/2302.06100/) (2023)
- [Can large language models build causal graphs?](https://mlsanity.com/paper/2303.05279/) (2023)
- [Explaining Legal Concepts with Augmented Large Language Models (GPT-4)](https://mlsanity.com/paper/2306.09525/) (2023)
- [Computer-Assisted Creation of Boolean Search Rules for Text Classification in the Legal Domain](https://mlsanity.com/paper/2112.05807/) (2021)
- [Sentence Embeddings and High-speed Similarity Search for Fast Computer Assisted Annotation of Legal Documents](https://mlsanity.com/paper/2112.11494/) (2021)
- [JusticeBot: A Methodology for Building Augmented Intelligence Tools for Laypeople to Increase Access to Justice](https://mlsanity.com/paper/2308.02032/) (2023)
- [LLMediator: GPT-4 Assisted Online Dispute Resolution](https://mlsanity.com/paper/2307.16732/) (2023)
- [Black-Box Analysis: GPTs Across Time in Legal Textual Entailment Task](https://mlsanity.com/paper/2309.05501/) (2023)
- [Beyond Logic Programming for Legal Reasoning](https://mlsanity.com/paper/2306.16632/) (2023)
- [Toward an Intelligent Tutoring System for Argument Mining in Legal Texts](https://mlsanity.com/paper/2210.13635/) (2022)

## Built on by (most cited first, 17 of 17)

- [Natural Language Processing for the Legal Domain: A Survey of Tasks, Datasets, Models, and Challenges](https://mlsanity.com/paper/2410.21306/) (2024)
- [AgentCourt: Simulating Court with Adversarial Evolvable Lawyer Agents](https://mlsanity.com/paper/2408.08089/) (2024)
- [Robots in the Middle: Evaluating LLMs in Dispute Resolution](https://mlsanity.com/paper/2410.07053/) (2024)
- [Large Language Models Meet Legal Artificial Intelligence: A Survey](https://mlsanity.com/paper/2509.09969/) (2025)
- [Getting in the Door: Streamlining Intake in Civil Legal Services with Large Language Models](https://mlsanity.com/paper/2410.03762/) (2024)
- [It Cannot Be Right If It Was Written by AI: On Lawyers' Preferences of Documents Perceived as Authored by an LLM vs a Human](https://mlsanity.com/paper/2407.06798/) (2024)
- [How will advanced AI systems impact democracy?](https://mlsanity.com/paper/2409.06729/) (2024)
- [Bye-bye, Bluebook? Automating Legal Drudgery With AI-Augmented Rule Following](https://mlsanity.com/paper/2505.02763/) (2025)
- [Weaving Pathways for Justice with GPT: LLM-driven automated drafting of interactive legal applications](https://mlsanity.com/paper/2312.09198/) (2023)
- [From Answers to Guidance: A Proactive Dialogue System for Legal Documents](https://mlsanity.com/paper/2510.19723/) (2025)
- [LegalWebAgent: Empowering Access to Justice via LLM-Based Web Agents](https://mlsanity.com/paper/2512.04105/) (2025)
- [From Legal Text to Executable Decision Models: Evaluating Structured Representations for Legal Decision Model Generation](https://mlsanity.com/paper/2604.17153/) (2026)
- [A Path Towards Legal Autonomy: An interoperable and explainable approach to extracting, transforming, loading and computing legal information using large language models, expert systems and Bayesian networks](https://mlsanity.com/paper/2403.18537/) (2024)
- [JUDGEBERT: Assessing Legal Meaning Preservation Between Sentences](https://mlsanity.com/paper/2508.16870/) (2025)
- [Mirror: A Multi-Agent System for AI-Assisted Ethics Review](https://mlsanity.com/paper/2602.13292/) (2026)
- [By Their Fruits You Will Know Them: Comparing Formalizations of Law by the Decisions They Encode](https://mlsanity.com/paper/2605.25186/) (2026)
- [From Judgments to Issues: Structured Extraction of Legal Reasoning with Citation-Hallucination Control](https://mlsanity.com/paper/2607.03325/) (2026)

## Abstract (arXiv)

Encoding legislative text in a formal representation is an important prerequisite to different tasks in the field of AI & Law. For example, rule-based expert systems focused on legislation can support laypeople in understanding how legislation applies to them and provide them with helpful context and information. However, the process of analyzing legislation and other sources to encode it in the desired formal representation can be time-consuming and represents a bottleneck in the development of such systems. Here, we investigate to what degree large language models (LLMs), such as GPT-4, are able to automatically extract structured representations from legislation. We use LLMs to create pathways from legislation, according to the JusticeBot methodology for legal decision support systems, evaluate the pathways and compare them to manually created pathways. The results are promising, with 60% of generated pathways being rated as equivalent or better than manually created ones in a blind comparison. The approach suggests a promising path to leverage the capabilities of LLMs to ease the costly development of systems based on symbolic approaches that are transparent and explainable.