PACT: Can Enterprise AI Assistants Be Trusted Under Pressure?
On the map: LLM agents and tool use → Agent threats, guardrails and governance
Abstract (arXiv)
As corporate AI adoption continues to grow, enterprise-grade LLM agents are being deployed into sensitive contexts such as hiring, healthcare, and finance. In these contexts, compliance with rules specified in an agent's system context is a first-order legal concern. Currently, no evaluation framework systematically measures which LLM models tend to violate compliance rules, especially under pressure from a persistent user, a hurried manager, or circumstances where violation is convenient or attractive. We introduce PACT (Pressure-Applied Compliance Testing), a benchmark for rule-following under pressure in AI agents assisting employees in daily tasks across twelve regulated enterprise domains and forty-eight scenarios, each set in a realistic multi-turn conversation. Each benchmark item pairs a standing rule against a rule-violating shortcut, and applies a battery of pressures across different wordings and system-prompt modes. We construct PACT component by component under strict LLM-as-judge auditing to ensure samples are unambiguous, ungameable, and realistic enough to avoid eliciting evaluation-aware behavior. We use PACT to profile LLM compliance across six complementary metrics that create a holistic picture of an AI assistant's robustness under pressure and throughout multi-turn conversations, its transparency, and ability to correctly discern where a rule applies. We aggregate this profile into PACTScore, a reliability-weighted compliance rate over all items and modes. Our results across 22 common LLM models spanning multiple providers and sizes show substantial variability in compliance across models and metric dimensions. Even the strongest assistants mis-apply a rule on 6 to 10% of items, and ordinary user pressure raises the violation rate by 65% on average. PACT highlights compliance risks in LLM assistants, motivating guardrails and careful model selection.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 19 of its 49 references that are in the map. First posted on arXiv 16 Sep 2026.
Builds on
19 of 49 references.
Closest ancestors · 1
Same direction, newest first
Other references · 18
- 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,429
- 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,402
- 2025gpt-oss-120b & gpt-oss-20b Model Cardcited by 1,386
- 2023Towards Understanding Sycophancy in Language Modelscited by 1,040
- 2024LLM Evaluators Recognize and Favor Their Own Generationscited by 686
- 2025τ2-Bench: Evaluating Conversational Agents in a Dual-Control Environmentcited by 627
- 2025LLMs Get Lost In Multi-Turn Conversationcited by 429
- 2024Frontier Models are Capable of In-context Schemingcited by 287
- 2024SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusalcited by 240
- 2023Large Language Models can Strategically Deceive their Users when Put Under Pressurecited by 146
- 2024Evaluating Frontier Models for Dangerous Capabilitiescited by 128
- 2025Large Language Models Often Know When They Are Being Evaluatedcited by 128
6 more
- 2024BetterBench: Assessing AI Benchmarks, Uncovering Issues, and Establishing Best Practicescited by 114
- 2023Are You Sure? Challenging LLMs Leads to Performance Drops in The FlipFlop Experimentcited by 77
- 2024AIR-Bench 2024: A Safety Benchmark Based on Risk Categories from Regulations and Policiescited by 74
- 2025IHEval: Evaluating Language Models on Following the Instruction Hierarchycited by 53
- 2025AIReg-Bench: Benchmarking Language Models That Assess AI Regulation Compliancecited by 6
- 2026CompliBench: Benchmarking LLM Judges for Compliance Violation Detection in Dialogue Systemscited by 6
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2025Evaluating LLM Agent Adherence to Hierarchical Safety Principles: A Lightweight Benchmark for Probing Foundational Controllability Components
- 2026COMPASS: A Framework for Evaluating Organization-Specific Policy Alignment in LLMs
- 2026The Compliance Gap: Why AI Systems Promise to Follow Process Instructions but Don't
- 2025Compliance Brain Assistant: Conversational Agentic AI for Assisting Compliance Tasks in Enterprise Environments
- 2026MANTRA: Synthesizing SMT-Validated Compliance Benchmarks for Tool-Using LLM Agents
- 2026From Obligation to Specification: A Survey on Validating EU AI Act Requirements in RE
- 2025Assertion-Conditioned Compliance: A Provenance-Aware Vulnerability in Multi-Turn Tool-Calling Agents
- 2026Auditing Provenance Sensitivity in LLM Agent Action Selection