Schematize: An Agentic System for Generating and Refining Information-Extraction Schemas for Legal Research
On the map: LLM agents and tool use → Social simulation and multi-agent debate
Abstract (arXiv)
Empirical legal research often relies on turning research questions into structured data extracted from large collections of rulings and judgments. Designing the extraction schema and then extracting the data remain a manual, expertise-heavy bottleneck. We present schematize, an open-source multi-agent system that interactively turns a researcher's problem statement into a validated extraction schema that can later be used for autonomous extraction. Schematize couples (i) a clarification dialogue that elicits implicit expert intent, (ii) iterative schema generation, (iii) data-grounded refinement that tests the schema against documents, and (iv) chat-based post-editing. We evaluated the system with human legal professional, introducing our novel methodology, and schematize achieves top performance in most of tested configurations. While the system is designed to be domain-agnostic and applicable to any document collection, we tailor and evaluate it on legal research problems. We release schematize as a pip-installable Python package with full documentation.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use. Builds on 12 of its 24 references that are in the map. First posted on arXiv 1 Sep 2026.
Builds on
12 of 24 references.
References · 12
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
- 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
- 2024Judging the Judges: Evaluating Alignment and Vulnerabilities in LLMs-as-Judgescited by 205
- 2023Schema-Driven Information Extraction from Heterogeneous Tablescited by 25
- 2025The Illusion of Progress: Re-evaluating Hallucination Detection in LLMscited by 22
- 2025LLMs4SchemaDiscovery: A Human-in-the-Loop Workflow for Scientific Schema Mining with Large Language Modelscited by 9
- 2025SynthTextEval: Synthetic Text Data Generation and Evaluation for High-Stakes Domainscited by 7
- 2025LOGOS: LLM-driven End-to-End Grounded Theory Development and Schema Induction for Qualitative Researchcited by 6
- 2025PARSE: LLM Driven Schema Optimization for Reliable Entity Extractioncited by 6
- 2025Intent-Aware Schema Generation And Refinement For Literature Review Tablescited by 5
- 2022Data-efficient End-to-end Information Extraction for Statistical Legal Analysiscited by 3
- 2026ScheMatiQ: From Research Question to Structured Data through Interactive Schema Discoverycited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026LawCompass: Navigating from Legal QA to Multi-Agent Deep Research with Grounded Evidence
- 2026LegalGraphRAG: Multi-Agent Graph Retrieval-Augmented Generation for Reliable Legal Reasoning
- 2026ExtractBench: A Benchmark for Schema-Guided Enterprise Document Extraction
- 2026Legal Research Bench: Measuring End-to-End Reliability in Long-Horizon Legal Research Agents
- 2026LawThinker: A Deep Research Legal Agent in Dynamic Environments
- 2026InvestigationWorlds: An Agentic Environment for Legal Investigation
- 2026L-MAD: A Systematic Evaluation of Multi-Agent Debate Structures in Legal Reasoning
- 2025PAKTON: A Multi-Agent Framework for Question Answering in Long Legal Agreements