# Analyzing Images of Legal Documents: Toward Multi-Modal LLMs for Access to Justice

arXiv: https://arxiv.org/abs/2412.15260 · HTML: https://mlsanity.com/paper/2412.15260/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 36 references that are in the map.
- First posted on arXiv 16 Dec 2024.

## Builds on (most cited first, 11 of 11)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Legal Prompting: Teaching a Language Model to Think Like a Lawyer](https://mlsanity.com/paper/2212.01326/) (2022)
- [Explaining Legal Concepts with Augmented Large Language Models (GPT-4)](https://mlsanity.com/paper/2306.09525/) (2023)
- [Blind Judgement: Agent-Based Supreme Court Modelling With GPT](https://mlsanity.com/paper/2301.05327/) (2023)
- [Unlocking Practical Applications in Legal Domain: Evaluation of GPT for Zero-Shot Semantic Annotation of Legal Texts](https://mlsanity.com/paper/2305.04417/) (2023)
- [Using Large Language Models to Support Thematic Analysis in Empirical Legal Studies](https://mlsanity.com/paper/2310.18729/) (2023)
- [Robots in the Middle: Evaluating LLMs in Dispute Resolution](https://mlsanity.com/paper/2410.07053/) (2024)
- [Black-Box Analysis: GPTs Across Time in Legal Textual Entailment Task](https://mlsanity.com/paper/2309.05501/) (2023)
- [Getting in the Door: Streamlining Intake in Civil Legal Services with Large Language Models](https://mlsanity.com/paper/2410.03762/) (2024)
- [Using LLMs to Discover Legal Factors](https://mlsanity.com/paper/2410.07504/) (2024)
- [Weaving Pathways for Justice with GPT: LLM-driven automated drafting of interactive legal applications](https://mlsanity.com/paper/2312.09198/) (2023)

## Built on by (most cited first, 4 of 4)

- [LegalWebAgent: Empowering Access to Justice via LLM-Based Web Agents](https://mlsanity.com/paper/2512.04105/) (2025)
- [Judging by Appearances? Auditing and Intervening Vision-Language Models for Bail Prediction](https://mlsanity.com/paper/2510.00088/) (2025)
- [Can You Trust What You See? Human and AI Detection of Synthetic Legal Evidence](https://mlsanity.com/paper/2606.07613/) (2026)
- [In With the Old: Enhancing 'Classical' Document Automation with Generative AI](https://mlsanity.com/paper/2610.07480/) (2026)

## Abstract (arXiv)

Interacting with the legal system and the government requires the assembly and analysis of various pieces of information that can be spread across different (paper) documents, such as forms, certificates and contracts (e.g. leases). This information is required in order to understand one's legal rights, as well as to fill out forms to file claims in court or obtain government benefits. However, finding the right information, locating the correct forms and filling them out can be challenging for laypeople. Large language models (LLMs) have emerged as a powerful technology that has the potential to address this gap, but still rely on the user to provide the correct information, which may be challenging and error-prone if the information is only available in complex paper documents. We present an investigation into utilizing multi-modal LLMs to analyze images of handwritten paper forms, in order to automatically extract relevant information in a structured format. Our initial results are promising, but reveal some limitations (e.g., when the image quality is low). Our work demonstrates the potential of integrating multi-modal LLMs to support laypeople and self-represented litigants in finding and assembling relevant information.