Automated Textbook Auditing with Multi-Agent LLM Systems
On the map: LLM agents and tool use → Web, search, and research agents
Abstract (arXiv)
Ensuring the quality of educational materials requires more than standard proofreading: textbooks must be audited for factual accuracy, domain-specific technical correctness, and linguistic quality simultaneously -- a task that general-purpose grammar checkers cannot address. We present AI Textbook Auditor, a modular multi-agent pipeline for automated quality assurance of educational materials across subject domains. The system accepts a textbook PDF and produces a structured, human-reviewable report via two analysis tracks: a Factual and Technical Track in which an ensemble of specialized LLM agents detects factual inaccuracies, code errors, incorrect definitions, and conceptual inconsistencies, augmented with web search for humanities domains; and a Grammar Track operating PDF-natively to preserve diacritical encoding. A Judge Agent filters false positives using domain-specific rules before presenting findings to a human reviewer. The pipeline supports two ingestion modes -- vision-native page rendering and PyMuPDF text extraction -- and is domain-adaptable via custom prompts encoding subject-specific error taxonomies. We demonstrate the system on two Romanian upper-secondary textbooks: a CS textbook (56 technical findings across seven categories, with an expert-validated precision of 62.5%) and a history and social sciences textbook (72 findings spanning factual errors, ideological bias, and grammar). The system is designed as a triage tool that reduces the manual effort of locating candidate issues, with human expert validation required before any editorial action.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use. Builds on 7 of its 9 references that are in the map. First posted on arXiv 13 Jul 2026.
Builds on
7 of 9 references.
References · 7
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
- 2021Evaluating Large Language Models Trained on Codecited by 8,053
- 2023Code Llama: Open Foundation Models for Codecited by 2,105
- 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,362
- 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,154
- 2022Grammatical Error Correction: A Survey of the State of the Artcited by 43
- 2024Can Large Language Models Automatically Score Proficiency of Written Essays?cited by 34
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Grading the Grader: Lessons from Evaluating an Agentic Data Analysis System
- 2025AI Agents-as-Judge: Automated Assessment of Accuracy, Consistency, Completeness and Clarity for Enterprise Documents
- 2026Auditing Automated Evaluation, Error Propagation, and Runtime Mitigation in Tool-Using Language Agents
- 2025The AI Committee: A Multi-Agent Framework for Automated Validation and Remediation of Web-Sourced Data
- 2026ScholarPeer: A Context-Aware Multi-Agent Framework for Automated Peer Review
- 2026An Agentic Evaluation Architecture for Historical Bias Detection in Educational Textbooks
- 2026Beyond the Text: Verifying That Agent-Written Papers Are Backed by Their Artifacts
- 2026Benchmarking Agentic Review Systems