An Agentic Evaluation Architecture for Historical Bias Detection in Educational Textbooks
On the map: LLM agents and tool use → Social simulation and multi-agent debate
Abstract (arXiv)
History textbooks often contain implicit biases, nationalist framing, and selective omissions that are difficult to audit at scale. We propose an agentic evaluation architecture comprising a multimodal screening agent, a heterogeneous jury of five evaluative agents, and a meta-agent for verdict synthesis and human escalation. A central contribution is a Source Attribution Protocol that distinguishes textbook narrative from quoted historical sources, preventing the misattribution that causes systematic false positives in single-model evaluators. In an empirical study on Romanian upper-secondary history textbooks, 83.3% of 270 screened excerpts were classified as pedagogically acceptable (mean severity 2.9/7), versus 5.4/7 under a zero-shot baseline, demonstrating that agentic deliberation mitigates over-penalization. In a blind human evaluation (18 evaluators, 54 comparisons), the Independent Deliberation configuration was preferred in 64.8% of cases over both a heuristic variant and the zero-shot baseline. At approximately 2 per textbook, these results position agentic evaluation architectures as economically viable decision-support tools for educational governance.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use. Builds on 13 of its 31 references that are in the map. First posted on arXiv 9 Apr 2026.
Builds on
13 of 31 references.
References · 13
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
- 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
- 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,205
- 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,179
- 2022Survey of Hallucination in Natural Language Generationcited by 2,056
- 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,906
- 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,739
- 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,270
- 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 695
- 2023AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviorscited by 580
- 2019LayoutLM: Pre-training of Text and Layout for Document Image Understandingcited by 549
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Automated Textbook Auditing with Multi-Agent LLM Systems
- 2025LLM Agents at the Roundtable: A Multi-Perspective and Dialectical Reasoning Framework for Essay Scoring
- 2026Grading the Grader: Lessons from Evaluating an Agentic Data Analysis System
- 2026MIRAGE: Auditing Anti-Muslim Bias in Frontier LLMs Across Reasoning, Agentic, and Time-Coupled Conditions
- 2025Structured Reasoning for Fairness: A Multi-Agent Approach to Bias Detection in Textual Data
- 2025When AIs Judge AIs: The Rise of Agent-as-a-Judge Evaluation for LLMs
- 2025The simulation of judgment in LLMs
- 2025Judging with Many Minds: Do More Perspectives Mean Less Prejudice? On Bias Amplifications and Resistance in Multi-Agent Based LLM-as-Judge