# IntelliAudit: Using Large Language Models to Evaluate Audit Controls

arXiv: https://arxiv.org/abs/2608.07688 · HTML: https://mlsanity.com/paper/2608.07688/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 27 references that are in the map.
- First posted on arXiv 7 Aug 2026.

## Builds on (most cited first, 17 of 17)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Improving Factuality and Reasoning in Language Models through Multiagent Debate](https://mlsanity.com/paper/2305.14325/) (2023)
- [LayoutLM: Pre-training of Text and Layout for Document Image Understanding](https://mlsanity.com/paper/1912.13318/) (2019)
- [LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking](https://mlsanity.com/paper/2204.08387/) (2022)
- [Multi-Agent Collaboration Mechanisms: A Survey of LLMs](https://mlsanity.com/paper/2501.06322/) (2025)
- [LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understanding](https://mlsanity.com/paper/2012.14740/) (2020)
- [Agentic Retrieval-Augmented Generation: A Survey on Agentic RAG](https://mlsanity.com/paper/2501.09136/) (2025)
- [Supporting Human-AI Collaboration in Auditing LLMs with LLMs](https://mlsanity.com/paper/2304.09991/) (2023)
- [LLM-Based Human-Agent Collaboration and Interaction Systems: A Survey](https://mlsanity.com/paper/2505.00753/) (2025)
- [NLP-based Automated Compliance Checking of Data Processing Agreements against GDPR](https://mlsanity.com/paper/2209.09722/) (2022)
- [Debate, Deliberate, Decide (D3): A Cost-Aware Adversarial Framework for Reliable and Interpretable LLM Evaluation](https://mlsanity.com/paper/2410.04663/) (2024)
- [GraphCompliance: Structure-Aware Regulatory Compliance through Policy-Context Graph Alignment](https://mlsanity.com/paper/2510.26309/) (2025)
- [LLMAuditor: A Framework for Auditing Large Language Models Using Human-in-the-Loop](https://mlsanity.com/paper/2402.09346/) (2024)
- [RAGulating Compliance: A Multi-Agent Knowledge Graph for Regulatory QA](https://mlsanity.com/paper/2508.09893/) (2025)
- [Smart Audit System Empowered by LLM](https://mlsanity.com/paper/2410.07677/) (2024)
- [Automating Security Audit Using Large Language Model based Agent: An Exploration Experiment](https://mlsanity.com/paper/2505.10732/) (2025)
- [AUDITFLOW: Executable Symbolic Environments for Structured Financial Reporting Verification](https://mlsanity.com/paper/2606.03031/) (2026)

## Abstract (arXiv)

IT audits require auditors to judge whether heterogeneous organizational evidence satisfies semantic security and compliance controls. This judgment is difficult to automate because relevant evidence is distributed across policies, records, spreadsheets, and operational artifacts, and because audit conclusions depend on evidentiary sufficiency rather than keyword matching. We present IntelliAudit, a retrieval-grounded multi-agent system for IT audit evidence evaluation. Given a control and an evidence corpus, IntelliAudit retrieves relevant artifacts, generates an evidence-grounded assessment, challenges adverse findings, adjudicates disagreements, and produces an auditor-facing recommendation with cited evidence, rationale, missing-evidence analysis, and remediation guidance. We instantiate IntelliAudit on ISO/IEC 27001 and evaluate it across multiple simulated organizations using expert auditor review and audit-readiness user feedback. The evaluation shows that IntelliAudit can support control interpretation, evidence-grounded reasoning, and audit-preparation workflows, while also revealing the importance of human oversight for calibrating sufficiency judgments and correcting overly permissive recommendations. These results suggest that retrieval-grounded multi-agent systems can assist audit evidence review, but should remain decision-support tools rather than autonomous certification systems.