Beyond Goodhart's Law: A Dynamic Benchmark for Evaluating Compliance in Multi-Agent Systems
On the map: LLM agents and tool use → Agent threats, guardrails and governance
Abstract (arXiv)
The rapid evolution of Large Language Models (LLMs) from passive assistants to autonomous, execution-capable agents has introduced critical operational risks. Most current evaluation frameworks neglect procedural compliance, leading to ''Machiavellian'' behaviors where agents strategically violate safety rules to maximize rewards - a direct manifestation of Goodhart's Law. To address this blind spot, we introduce MAC-Bench, a dynamic, adversarial benchmark designed to evaluate the procedural alignment of multi-agent systems under realistic pressure. We propose the SERV(Seed - Evolve - Refine - Verify) pipeline, an ``Agent-as-a-Benchmark'' paradigm that transforms unstructured legal texts into executable, contamination-free scenarios. By synthesizing holographic sandbox environments and injecting calibrated social-engineering pressure vectors, MAC-Bench forces agents into Pareto-optimal trade-offs between task success and regulatory adherence. We introduced novel metrics: the Compliance-Weighted Success Rate (CSR) and the Machiavellian Gap (MG), and conducted a comprehensive evaluation of state-of-the-art frontier models to reveal the pervasive trade-offs between success and compliance.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 21 of its 76 references that are in the map. First posted on arXiv 5 Jun 2026.
Builds on
21 of 76 references.
Closest ancestors · 2
Same direction, newest first
- 2025MAGPIE: A benchmark for Multi-AGent contextual PrIvacy Evaluation16 Oct
- 2024AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Bases17 Jul
Other references · 19
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
- 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,276
- 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,988
- 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,947
- 2016Concrete Problems in AI Safetycited by 1,642
- 2023GAIA: a benchmark for General AI Assistantscited by 1,128
- 2023ChatDev: Communicative Agents for Software Developmentcited by 919
- 2024LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methodscited by 465
- 2023Do the Rewards Justify the Means? Measuring Trade-Offs Between Rewards and Ethical Behavior in the MACHIAVELLI Benchmarkcited by 182
- 2024PrivacyLens: Evaluating Privacy Norm Awareness of Language Models in Actioncited by 171
- 2025Evaluation and Benchmarking of LLM Agents: A Surveycited by 152
7 more
- 2024Natural Language Processing for the Legal Domain: A Survey of Tasks, Datasets, Models, and Challengescited by 42
- 2024PsychoGAT: A Novel Psychological Measurement Paradigm through Interactive Fiction Games with LLM Agentscited by 28
- 2024Inference-Time Decontamination: Reusing Leaked Benchmarks for Large Language Model Evaluationcited by 19
- 2026Adversarial Reward Auditing for Active Detection and Mitigation of Reward Hackingcited by 11
- 2025CORE: Full-Path Evaluation of LLM Agents Beyond Final Statecited by 9
- 2025End-to-End Optimization of LLM-Driven Multi-Agent Search Systems via Heterogeneous-Group-Based Reinforcement Learningcited by 8
- 2023A Case Study for Compliance as Code with Graphs and Language Models: Public release of the Regulatory Knowledge Graphcited by 4
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026BLINDSPOT: A Benchmark for Safety and Refusal Calibration in Long-Horizon Tool-Using Agents
- 2025EU-Agent-Bench: Measuring Illegal Behavior of LLM Agents Under EU Law
- 2025AgenticEval: Toward Agentic and Self-Evolving Safety Evaluation of Large Language Models
- 2026AgentHijack: Benchmarking Computer Use Agent Robustness to Common Environment Corruptions
- 2026MANTRA: Synthesizing SMT-Validated Compliance Benchmarks for Tool-Using LLM Agents
- 2026AgentBoundary: Counterfactual Evaluation of Safety in Tool-Using LLM Agents
- 2026A Unified Evaluation Framework for Trustworthy Large Language Models, Agentic AI, and Multimodal Systems
- 2025TAMAS: Benchmarking Adversarial Risks in Multi-Agent LLM Systems