# The Hidden Structure -- Improving Legal Document Understanding Through Explicit Text Formatting

arXiv: https://arxiv.org/abs/2505.12837 · HTML: https://mlsanity.com/paper/2505.12837/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 46 references that are in the map.
- First posted on arXiv 19 May 2025.

## Builds on (most cited first, 23 of 23)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [Let's Verify Step by Step](https://mlsanity.com/paper/2305.20050/) (2023)
- [A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applications](https://mlsanity.com/paper/2402.07927/) (2024)
- [Ragas: Automated Evaluation of Retrieval Augmented Generation](https://mlsanity.com/paper/2309.15217/) (2023)
- [LegalBench: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Models](https://mlsanity.com/paper/2308.11462/) (2023)
- [CUAD: An Expert-Annotated NLP Dataset for Legal Contract Review](https://mlsanity.com/paper/2103.06268/) (2021)
- [Does Prompt Formatting Have Any Impact on LLM Performance?](https://mlsanity.com/paper/2411.10541/) (2024)
- [Large Language Models in Law: A Survey](https://mlsanity.com/paper/2312.03718/) (2023)
- [The Prompt Report: A Systematic Survey of Prompt Engineering Techniques](https://mlsanity.com/paper/2406.06608/) (2024)
- [LegalBench-RAG: A Benchmark for Retrieval-Augmented Generation in the Legal Domain](https://mlsanity.com/paper/2408.10343/) (2024)
- [Efficient multi-prompt evaluation of LLMs](https://mlsanity.com/paper/2405.17202/) (2024)
- [PromptBench: A Unified Library for Evaluation of Large Language Models](https://mlsanity.com/paper/2312.07910/) (2023)
- [Better Call GPT, Comparing Large Language Models Against Lawyers](https://mlsanity.com/paper/2401.16212/) (2024)
- [Struct-X: Enhancing Large Language Models Reasoning with Structured Data](https://mlsanity.com/paper/2407.12522/) (2024)
- [Notes on Applicability of GPT-4 to Document Understanding](https://mlsanity.com/paper/2405.18433/) (2024)
- [MDEval: Evaluating and Enhancing Markdown Awareness in Large Language Models](https://mlsanity.com/paper/2501.15000/) (2025)
- [A Law Reasoning Benchmark for LLM with Tree-Organized Structures including Factum Probandum, Evidence and Experiences](https://mlsanity.com/paper/2503.00841/) (2025)
- [The Law of Large Documents: Understanding the Structure of Legal Contracts Using Visual Cues](https://mlsanity.com/paper/2107.08128/) (2021)
- [Effect of Selection Format on LLM Performance](https://mlsanity.com/paper/2503.06926/) (2025)

## Built on by (most cited first, 5 of 5)

- [Benchmarking Small Language Models and Small Reasoning Language Models on System Log Severity Classification](https://mlsanity.com/paper/2601.07790/) (2026)
- [Agent-Based Software Artifact Evaluation](https://mlsanity.com/paper/2602.02235/) (2026)
- [MIRAGE: A Micro-Interaction Relational Architecture for Grounded Exploration in Multi-Figure Artworks](https://mlsanity.com/paper/2604.23788/) (2026)
- [Information Extraction from Electricity Invoices with General-Purpose Large Language Models](https://mlsanity.com/paper/2604.25927/) (2026)
- [ContraVis: Evidence-Grounded Visual Analytics for Contradiction Review in Legal Contracts](https://mlsanity.com/paper/2609.27014/) (2026)

## Abstract (arXiv)

Legal contracts possess an inherent, semantically vital structure (e.g., sections, clauses) that is crucial for human comprehension but whose impact on LLM processing remains under-explored. This paper investigates the effects of explicit input text structure and prompt engineering on the performance of GPT-4o and GPT-4.1 on a legal question-answering task using an excerpt of the CUAD. We compare model exact-match accuracy across various input formats: well-structured plain-text (human-generated from CUAD), plain-text cleaned of line breaks, extracted plain-text from Azure OCR, plain-text extracted by GPT-4o Vision, and extracted (and interpreted) Markdown (MD) from GPT-4o Vision. To give an indication of the impact of possible prompt engineering, we assess the impact of shifting task instructions to the system prompt and explicitly informing the model about the structured nature of the input. Our findings reveal that GPT-4o demonstrates considerable robustness to variations in input structure, but lacks in overall performance. Conversely, GPT-4.1's performance is markedly sensitive; poorly structured inputs yield suboptimal results (but identical with GPT-4o), while well-structured formats (original CUAD text, GPT-4o Vision text and GPT-4o MD) improve exact-match accuracy by ~20 percentage points. Optimizing the system prompt to include task details and an advisory about structured input further elevates GPT-4.1's accuracy by an additional ~10-13 percentage points, with Markdown ultimately achieving the highest performance under these conditions (79 percentage points overall exact-match accuracy). This research empirically demonstrates that while newer models exhibit greater resilience, careful input structuring and strategic prompt design remain critical for optimizing the performance of LLMs, and can significantly affect outcomes in high-stakes legal applications.