# PARSE: LLM Driven Schema Optimization for Reliable Entity Extraction

arXiv: https://arxiv.org/abs/2510.08623 · HTML: https://mlsanity.com/paper/2510.08623/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in Tool learning and function calling, one of 16 directions in LLM agents and tool use.
- Builds on 13 of its 19 references that are in the map.
- First posted on arXiv 8 Oct 2025.

## Placed in

- LLM agents and tool use / [Tool learning and function calling](https://mlsanity.com/agents/tool-learning-and-function-calling/)

## Builds on (most cited first, 13 of 13)

- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Towards Scalable Multi-domain Conversational Agents: The Schema-Guided Dialogue Dataset](https://mlsanity.com/paper/1909.05855/) (2019)
- [AgentSpec: Customizable Runtime Enforcement for Safe and Reliable LLM Agents](https://mlsanity.com/paper/2503.18666/) (2025)
- [Large Language Model Agent: A Survey on Methodology, Applications and Challenges](https://mlsanity.com/paper/2503.21460/) (2025)
- [Learning to Generate Structured Output with Schema Reinforcement Learning](https://mlsanity.com/paper/2502.18878/) (2025)
- [LLM-Powered AI Agent Systems and Their Applications in Industry](https://mlsanity.com/paper/2505.16120/) (2025)
- [Think Inside the JSON: Reinforcement Strategy for Strict LLM Schema Adherence](https://mlsanity.com/paper/2502.14905/) (2025)
- [Embedding Self-Correction as an Inherent Ability in Large Language Models for Enhanced Mathematical Reasoning](https://mlsanity.com/paper/2410.10735/) (2024)
- [Position: The Real Barrier to LLM Agent Usability is Agentic ROI](https://mlsanity.com/paper/2505.17767/) (2025)
- [Information Extraction in Low-Resource Scenarios: Survey and Perspective](https://mlsanity.com/paper/2202.08063/) (2022)
- [MARCO: Multi-Agent Real-time Chat Orchestration](https://mlsanity.com/paper/2410.21784/) (2024)
- [Improving the Reliability of LLMs: Combining CoT, RAG, Self-Consistency, and Self-Verification](https://mlsanity.com/paper/2505.09031/) (2025)
- [General LLMs as Instructors for Domain-Specific LLMs: A Sequential Fusion Method to Integrate Extraction and Editing](https://mlsanity.com/paper/2403.15736/) (2024)

## Built on by (most cited first, 6 of 6)

- [The Convergence of Schema-Guided Dialogue Systems and the Model Context Protocol](https://mlsanity.com/paper/2602.18764/) (2026)
- [Formal Semantics for Agentic Tool Protocols: A Process Calculus Approach](https://mlsanity.com/paper/2603.24747/) (2026)
- [Making Failure Safe: A Constrained, Verifiable Agent Framework for Open-Web Data Collection](https://mlsanity.com/paper/2607.00035/) (2026)
- [Discourse-Aware Policy Analysis with Argumentation: A Hybrid LLM-Symbolic Framework for Disaster Governance](https://mlsanity.com/paper/2607.13260/) (2026)
- [Schematize: An Agentic System for Generating and Refining Information-Extraction Schemas for Legal Research](https://mlsanity.com/paper/2609.22209/) (2026)
- [Adapt Semantics, Not Structure: Few-Instance Schema Calibration for Scientific PDF Extraction](https://mlsanity.com/paper/2609.34841/) (2026)

## Abstract (arXiv)

Structured information extraction from unstructured text is critical for emerging Software 3.0 systems where LLM agents autonomously interact with APIs and tools. Recent approaches apply large language models directly to extraction tasks using existing JSON schemas, often with constraint decoding or reinforcement learning approaches to ensure syntactic validity, but treat JSON schemas as static contracts designed for human developers, leading to suboptimal extraction performance, frequent hallucinations, and unreliable agent behavior when schemas contain ambiguous or incomplete specifications. We recognize that JSON schemas themselves are a form of natural language understanding contract that encodes rules, relationships, and expectations about data structure contracts that LLMs should be able to both interpret and systematically improve. Consequently, we develop PARSE (Parameter Automated Refinement and Schema Extraction), a novel system with two synergistic components: ARCHITECT, which autonomously optimizes JSON schemas for LLM consumption while maintaining backward compatibility through RELAY (an integrated code generation system), and SCOPE, which implements reflection-based extraction with combined static and LLM-based guardrails. We evaluate PARSE qualitatively and quantitatively on three datasets including Schema-Guided Dialogue (SGD), Structured Web Data Extraction (SWDE), and internal retail conversation data, and find that it achieves up to 64.7% improvement in extraction accuracy on SWDE with combined framework improvements reaching 10% across models, while reducing extraction errors by 92% within the first retry and and maintaining practical latency.