# PAKTON: A Multi-Agent Framework for Question Answering in Long Legal Agreements

arXiv: https://arxiv.org/abs/2506.00608 · HTML: https://mlsanity.com/paper/2506.00608/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 79 references that are in the map.
- First posted on arXiv 31 May 2025.

## Builds on (most cited first, 30 of 32)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment](https://mlsanity.com/paper/2303.16634/) (2023)
- [Gemma 3 Technical Report](https://mlsanity.com/paper/2503.19786/) (2025)
- [MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers](https://mlsanity.com/paper/2002.10957/) (2020)
- [From Local to Global: A Graph RAG Approach to Query-Focused Summarization](https://mlsanity.com/paper/2404.16130/) (2024)
- [Can Large Language Models Be an Alternative to Human Evaluations?](https://mlsanity.com/paper/2305.01937/) (2023)
- [CUAD: An Expert-Annotated NLP Dataset for Legal Contract Review](https://mlsanity.com/paper/2103.06268/) (2021)
- [ContractNLI: A Dataset for Document-level Natural Language Inference for Contracts](https://mlsanity.com/paper/2110.01799/) (2021)
- [SaulLM-7B: A pioneering Large Language Model for Law](https://mlsanity.com/paper/2403.03883/) (2024)
- [Large Language Models in Law: A Survey](https://mlsanity.com/paper/2312.03718/) (2023)
- [LegalBench-RAG: A Benchmark for Retrieval-Augmented Generation in the Legal Domain](https://mlsanity.com/paper/2408.10343/) (2024)
- [CBR-RAG: Case-Based Reasoning for Retrieval Augmented Generation in LLMs for Legal Question Answering](https://mlsanity.com/paper/2404.04302/) (2024)
- [LLM Comparative Assessment: Zero-shot NLG Evaluation through Pairwise Comparisons using Large Language Models](https://mlsanity.com/paper/2307.07889/) (2023)
- [Question Answering for Privacy Policies: Combining Computational and Legal Perspectives](https://mlsanity.com/paper/1911.00841/) (2019)
- [Into the Unknown Unknowns: Engaged Human Learning through Participation in Language Model Agent Conversations](https://mlsanity.com/paper/2408.15232/) (2024)
- [SaulLM-54B & SaulLM-141B: Scaling Up Domain Adaptation for the Legal Domain](https://mlsanity.com/paper/2407.19584/) (2024)
- [LawLLM: Law Large Language Model for the US Legal System](https://mlsanity.com/paper/2407.21065/) (2024)
- [LumberChunker: Long-Form Narrative Document Segmentation](https://mlsanity.com/paper/2406.17526/) (2024)
- [MAUD: An Expert-Annotated Legal NLP Dataset for Merger Agreement Understanding](https://mlsanity.com/paper/2301.00876/) (2023)
- [HM-RAG: Hierarchical Multi-Agent Multimodal Retrieval Augmented Generation](https://mlsanity.com/paper/2504.12330/) (2025)
- [Towards Robust Legal Reasoning: Harnessing Logical LLMs in Law](https://mlsanity.com/paper/2502.17638/) (2025)
- [Multi-Agent Simulator Drives Language Models for Legal Intensive Interaction](https://mlsanity.com/paper/2502.06882/) (2025)
- [RAG based Question-Answering for Contextual Response Prediction System](https://mlsanity.com/paper/2409.03708/) (2024)
- [EchoPrompt: Instructing the Model to Rephrase Queries for Improved In-context Learning](https://mlsanity.com/paper/2309.10687/) (2023)
- [QuIM-RAG: Advancing Retrieval-Augmented Generation with Inverted Question Matching for Enhanced QA Performance](https://mlsanity.com/paper/2501.02702/) (2025)

## Built on by (most cited first, 1 of 1)

- [LLM Agents in Law: Taxonomy, Applications, and Challenges](https://mlsanity.com/paper/2601.06216/) (2026)

## Abstract (arXiv)

Contract review is a complex and time-intensive task that typically demands specialized legal expertise, rendering it largely inaccessible to non-experts. Moreover, legal interpretation is rarely straightforward-ambiguity is pervasive, and judgments often hinge on subjective assessments. Compounding these challenges, contracts are usually confidential, restricting their use with proprietary models and necessitating reliance on open-source alternatives. To address these challenges, we introduce PAKTON: a fully open-source, end-to-end, multi-agent framework with plug-and-play capabilities. PAKTON is designed to handle the complexities of contract analysis through collaborative agent workflows and a novel retrieval-augmented generation (RAG) component, enabling automated legal document review that is more accessible, adaptable, and privacy-preserving. Experiments demonstrate that PAKTON outperforms both general-purpose and pretrained models in predictive accuracy, retrieval performance, explainability, completeness, and grounded justifications as evaluated through a human study and validated with automated metrics.