mlsanityData to 7 Oct 2026

AEMA: Verifiable Evaluation Framework for Trustworthy and Controlled Agentic LLM Systems

arXiv 2601.11903 ↗First posted 17 Jan 2026cs.AIcited by 1

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

Evaluating large language model (LLM)-based multi-agent systems remains a critical challenge, as these systems must exhibit reliable coordination, transparent decision-making, and verifiable performance across evolving tasks. Existing evaluation approaches often limit themselves to single-response scoring or narrow benchmarks, which lack stability, extensibility, and automation when deployed in enterprise settings at multi-agent scale. We present AEMA (Adaptive Evaluation Multi-Agent), a process-aware and auditable framework that plans, executes, and aggregates multi-step evaluations across heterogeneous agentic workflows under human oversight. Compared to a single LLM-as-a-Judge, AEMA achieves greater stability, human alignment, and traceable records that support accountable automation. Our results on enterprise-style agent workflows simulated using realistic business scenarios demonstrate that AEMA provides a transparent and reproducible pathway toward responsible evaluation of LLM-based multi-agent systems. Keywords Agentic AI, Multi-Agent Systems, Trustworthy AI, Verifiable Evaluation, Human Oversight

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 15 of its 25 references that are in the map. First posted on arXiv 17 Jan 2026.

Builds on

15 of 25 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Agent-as-a-Judge: Evaluate Agents with Agents14 Oct
  2. 2023ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debate14 Aug

Other references · 13

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  2. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,205
  3. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,906
  4. 2023Large Language Models Are State-of-the-Art Evaluators of Translation Qualitycited by 416
  5. 2023Evaluating Large Language Models at Evaluating Instruction Followingcited by 290
  6. 2021The Perils of Using Mechanical Turk to Evaluate Open-Ended Text Generationcited by 96
  7. 2025Mind2Web 2: Evaluating Agentic Search with Agent-as-a-Judgecited by 75
  8. 2023Self-ICL: Zero-Shot In-Context Learning with Self-Generated Demonstrationscited by 42
  9. 2025BEARCUBS: A benchmark for computer-using web agentscited by 29
  10. 2023Auto-ICL: In-Context Learning without Human Supervisioncited by 18
  11. 2024Sparse Meets Dense: A Hybrid Approach to Enhance Scientific Document Retrievalcited by 16
  12. 2024Can Many-Shot In-Context Learning Help LLMs as Evaluators? A Preliminary Empirical Studycited by 15
1 more
  1. 2019Kendall Tau Sequence Distance: Extending Kendall Tau from Ranks to Sequencescited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026From Accuracy to Auditability: A Survey of Determinism in Financial AI Systemscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Agentic CLEAR: Automating Multi-Level Evaluation of LLM Agents
  2. 2025VeriLA: A Human-Centered Evaluation Framework for Interpretable Verification of LLM Agent Failures
  3. 2025Evaluation and Benchmarking of LLM Agents: A Survey
  4. 2026A Unified Evaluation Framework for Trustworthy Large Language Models, Agentic AI, and Multimodal Systems
  5. 2026Agent-as-a-Judge
  6. 2024Evaluation-Driven Development and Operations of LLM Agents: A Process Model and Reference Architecture
  7. 2026AgentCompass: A Unified Evaluation Infrastructure for Agent Capabilities
  8. 2025Empowering Real-World: A Survey on the Technology, Practice, and Evaluation of LLM-driven Industry Agents