# Knowledge Graph Representations for LLM-Based Policy Compliance Reasoning

arXiv: https://arxiv.org/abs/2604.27713 · HTML: https://mlsanity.com/paper/2604.27713/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning.
- Builds on 19 of its 41 references that are in the map.
- First posted on arXiv 30 Apr 2026.

## Placed in

- LLM reasoning / [Reasoning for agents, planning and code](https://mlsanity.com/lr/reasoning-for-agents-planning-and-code/)

## Builds on (most cited first, 19 of 19)

- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [From Local to Global: A Graph RAG Approach to Query-Focused Summarization](https://mlsanity.com/paper/2404.16130/) (2024)
- [LegalBench: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Models](https://mlsanity.com/paper/2308.11462/) (2023)
- [Reasoning on Graphs: Faithful and Interpretable Large Language Model Reasoning](https://mlsanity.com/paper/2310.01061/) (2023)
- [Think-on-Graph: Deep and Responsible Reasoning of Large Language Model on Knowledge Graph](https://mlsanity.com/paper/2307.07697/) (2023)
- [LLMs for Knowledge Graph Construction and Reasoning: Recent Capabilities and Future Opportunities](https://mlsanity.com/paper/2305.13168/) (2023)
- [Simple Is Effective: The Roles of Graphs and Large Language Models in Knowledge-Graph-Based Retrieval-Augmented Generation](https://mlsanity.com/paper/2410.20724/) (2024)
- [KGGen: Extracting Knowledge Graphs from Plain Text with Language Models](https://mlsanity.com/paper/2502.09956/) (2025)
- [PolicyQA: A Reading Comprehension Dataset for Privacy Policies](https://mlsanity.com/paper/2010.02557/) (2020)
- [Large Language Models Meet Knowledge Graphs for Question Answering: Synthesis and Opportunities](https://mlsanity.com/paper/2505.20099/) (2025)
- [Large Language Models Can Better Understand Knowledge Graphs Than We Thought](https://mlsanity.com/paper/2402.11541/) (2024)
- [Automated Knowledge Graph Construction using Large Language Models and Sentence Complexity Modelling](https://mlsanity.com/paper/2509.17289/) (2025)
- [GraphCompliance: Structure-Aware Regulatory Compliance through Policy-Context Graph Alignment](https://mlsanity.com/paper/2510.26309/) (2025)
- [Thinking with Knowledge Graphs: Enhancing LLM Reasoning Through Structured Data](https://mlsanity.com/paper/2412.10654/) (2024)
- [KG-LLM-Bench: A Scalable Benchmark for Evaluating LLM Reasoning on Textualized Knowledge Graphs](https://mlsanity.com/paper/2504.07087/) (2025)
- [RAGulating Compliance: A Multi-Agent Knowledge Graph for Regulatory QA](https://mlsanity.com/paper/2508.09893/) (2025)
- [PrivComp-KG : Leveraging Knowledge Graph and Large Language Models for Privacy Policy Compliance Verification](https://mlsanity.com/paper/2404.19744/) (2024)
- [Let's have a chat with the EU AI Act](https://mlsanity.com/paper/2505.11946/) (2025)

## Built on by (most cited first, 2 of 2)

- [EADC: Evaluation of Advanced and Deep-level Compliance in Large Language Models](https://mlsanity.com/paper/2609.26175/) (2026)
- [LLM-Based Policy Compliance Verification of Loan Applications](https://mlsanity.com/paper/2610.20155/) (2026)

## Abstract (arXiv)

The risks posed by AI features are increasing as they are rapidly integrated into software applications. In response, regulations and standards for safe and secure AI have been proposed. In this paper, we present an agentic framework that constructs knowledge graphs (KGs) from AI policy documents and retrieves policy-relevant information to answer questions. We build KGs from three AI risk-related polices under two ontology schemas, and then evaluate five LLMs on 42 policy QA tasks spanning six reasoning types, from entity lookup to cross-policy inference, using both heuristic scoring and an LLM-as-judge. KG augmentation improves scores for all five models, and an open, LLM-discovered schema matches or exceeds the formal ontology.