# Schematize: An Agentic System for Generating and Refining Information-Extraction Schemas for Legal Research

arXiv: https://arxiv.org/abs/2609.22209 · HTML: https://mlsanity.com/paper/2609.22209/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use.
- Builds on 12 of its 24 references that are in the map.
- First posted on arXiv 1 Sep 2026.

## Placed in

- LLM agents and tool use / [Social simulation and multi-agent debate](https://mlsanity.com/agents/social-simulation-and-multi-agent-debate/)

## Builds on (most cited first, 12 of 12)

- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [Judging the Judges: Evaluating Alignment and Vulnerabilities in LLMs-as-Judges](https://mlsanity.com/paper/2406.12624/) (2024)
- [Schema-Driven Information Extraction from Heterogeneous Tables](https://mlsanity.com/paper/2305.14336/) (2023)
- [The Illusion of Progress: Re-evaluating Hallucination Detection in LLMs](https://mlsanity.com/paper/2508.08285/) (2025)
- [LLMs4SchemaDiscovery: A Human-in-the-Loop Workflow for Scientific Schema Mining with Large Language Models](https://mlsanity.com/paper/2504.00752/) (2025)
- [SynthTextEval: Synthetic Text Data Generation and Evaluation for High-Stakes Domains](https://mlsanity.com/paper/2507.07229/) (2025)
- [LOGOS: LLM-driven End-to-End Grounded Theory Development and Schema Induction for Qualitative Research](https://mlsanity.com/paper/2509.24294/) (2025)
- [PARSE: LLM Driven Schema Optimization for Reliable Entity Extraction](https://mlsanity.com/paper/2510.08623/) (2025)
- [Intent-Aware Schema Generation And Refinement For Literature Review Tables](https://mlsanity.com/paper/2507.19521/) (2025)
- [Data-efficient End-to-end Information Extraction for Statistical Legal Analysis](https://mlsanity.com/paper/2211.01692/) (2022)
- [ScheMatiQ: From Research Question to Structured Data through Interactive Schema Discovery](https://mlsanity.com/paper/2604.09237/) (2026)

## Abstract (arXiv)

Empirical legal research often relies on turning research questions into structured data extracted from large collections of rulings and judgments. Designing the extraction schema and then extracting the data remain a manual, expertise-heavy bottleneck. We present schematize, an open-source multi-agent system that interactively turns a researcher's problem statement into a validated extraction schema that can later be used for autonomous extraction. Schematize couples (i) a clarification dialogue that elicits implicit expert intent, (ii) iterative schema generation, (iii) data-grounded refinement that tests the schema against documents, and (iv) chat-based post-editing. We evaluated the system with human legal professional, introducing our novel methodology, and schematize achieves top performance in most of tested configurations. While the system is designed to be domain-agnostic and applicable to any document collection, we tailor and evaluate it on legal research problems. We release schematize as a pip-installable Python package with full documentation.