# HarmonyGuard: Toward Safety and Utility in Web Agents via Adaptive Policy Enhancement and Dual-Objective Optimization

arXiv: https://arxiv.org/abs/2508.04010 · HTML: https://mlsanity.com/paper/2508.04010/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use.
- Builds on 25 of its 28 references that are in the map.
- First posted on arXiv 6 Aug 2025.

## Placed in

- LLM agents and tool use / [Agent threats, guardrails and governance](https://mlsanity.com/agents/agent-threats-guardrails-and-governance/)

## Builds on (most cited first, 25 of 25)

- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [Qwen Technical Report](https://mlsanity.com/paper/2309.16609/) (2023)
- [WebArena: A Realistic Web Environment for Building Autonomous Agents](https://mlsanity.com/paper/2307.13854/) (2023)
- [Instruction Tuning for Large Language Models: A Survey](https://mlsanity.com/paper/2308.10792/) (2023)
- [The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions](https://mlsanity.com/paper/2404.13208/) (2024)
- [AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Bases](https://mlsanity.com/paper/2407.12784/) (2024)
- [WASP: Benchmarking Web Agent Security Against Prompt Injection Attacks](https://mlsanity.com/paper/2504.18575/) (2025)
- [Robust Prompt Optimization for Defending Language Models Against Jailbreaking Attacks](https://mlsanity.com/paper/2401.17263/) (2024)
- [ShieldAgent: Shielding Agents via Verifiable Safety Policy Reasoning](https://mlsanity.com/paper/2503.22738/) (2025)
- [EIA: Environmental Injection Attack on Generalist Web Agents for Privacy Leakage](https://mlsanity.com/paper/2409.11295/) (2024)
- [InfiGUI-R1: Advancing Multimodal GUI Agents from Reactive Actors to Deliberative Reasoners](https://mlsanity.com/paper/2504.14239/) (2025)
- [ST-WebAgentBench: A Benchmark for Evaluating Safety and Trustworthiness in Web Agents](https://mlsanity.com/paper/2410.06703/) (2024)
- [LlamaFirewall: An open source guardrail system for building secure AI agents](https://mlsanity.com/paper/2505.03574/) (2025)
- [GuardAgent: Safeguard LLM Agents by a Guard Agent via Knowledge-Enabled Reasoning](https://mlsanity.com/paper/2406.09187/) (2024)
- [Dissecting Adversarial Robustness of Multimodal LM Agents](https://mlsanity.com/paper/2406.12814/) (2024)
- [Attacking Vision-Language Computer Agents via Pop-ups](https://mlsanity.com/paper/2411.02391/) (2024)
- [The Task Shield: Enforcing Task Alignment to Defend Against Indirect Prompt Injection in LLM Agents](https://mlsanity.com/paper/2412.16682/) (2024)
- [InfiGUIAgent: A Multimodal Generalist GUI Agent with Native Reasoning and Reflection](https://mlsanity.com/paper/2501.04575/) (2025)
- [Refusal-Trained LLMs Are Easily Jailbroken As Browser Agents](https://mlsanity.com/paper/2410.13886/) (2024)
- [Metacognitive Prompting Improves Understanding in Large Language Models](https://mlsanity.com/paper/2308.05342/) (2023)
- [WIPI: A New Web Threat for LLM-Driven Web Agents](https://mlsanity.com/paper/2402.16965/) (2024)
- [Feedback-Guided Extraction of Knowledge Base from Retrieval-Augmented LLM Applications](https://mlsanity.com/paper/2411.14110/) (2024)
- [Evaluating the Robustness of Multimodal Agents Against Active Environmental Injection Attacks](https://mlsanity.com/paper/2502.13053/) (2025)
- [Can We Predict Alignment Before Models Finish Thinking? Towards Monitoring Misaligned Reasoning Models](https://mlsanity.com/paper/2507.12428/) (2025)
- [Think Twice Before You Act: Enhancing Agent Behavioral Safety with Thought Correction](https://mlsanity.com/paper/2505.11063/) (2025)

## Built on by (most cited first, 10 of 10)

- [MemEye: A Visual-Centric Evaluation Framework for Multimodal Agent Memory](https://mlsanity.com/paper/2605.15128/) (2026)
- [SafePred: A Predictive Guardrail for Computer-Using Agents via World Models](https://mlsanity.com/paper/2602.01725/) (2026)
- [ColorBench: Benchmarking Mobile Agents with Graph-Structured Framework for Complex Long-Horizon Tasks](https://mlsanity.com/paper/2510.14621/) (2025)
- [GUI-PRA: Process Reward Agent for GUI Tasks](https://mlsanity.com/paper/2509.23263/) (2025)
- [Graph2Eval: Automatic Multimodal Task Generation for Agents via Knowledge Graphs](https://mlsanity.com/paper/2510.00507/) (2025)
- [Your Agentic LLMs Secretly Encode Indirect Prompt-Injection Exposure in Hidden States](https://mlsanity.com/paper/2608.02657/) (2026)
- [SnapGuard: Lightweight Prompt Injection Detection for Screenshot-Based Web Agents](https://mlsanity.com/paper/2604.25562/) (2026)
- [Learning Efficient Guardrails for Compliance](https://mlsanity.com/paper/2510.03485/) (2025)
- [SkillHarness: Harnessing Safe Skills for Computer-Use Agents](https://mlsanity.com/paper/2606.20636/) (2026)
- [Policy Compliance of User Requests in Natural Language for AI Systems](https://mlsanity.com/paper/2603.00369/) (2026)

## Abstract (arXiv)

Large language models enable agents to autonomously perform tasks in open web environments. However, as hidden threats within the web evolve, web agents face the challenge of balancing task performance with emerging risks during long-sequence operations. Although this challenge is critical, current research remains limited to single-objective optimization or single-turn scenarios, lacking the capability for collaborative optimization of both safety and utility in web environments. To address this gap, we propose HarmonyGuard, a multi-agent collaborative framework that leverages policy enhancement and objective optimization to jointly improve both utility and safety. HarmonyGuard features a multi-agent architecture characterized by two fundamental capabilities: (1) Adaptive Policy Enhancement: We introduce the Policy Agent within HarmonyGuard, which automatically extracts and maintains structured security policies from unstructured external documents, while continuously updating policies in response to evolving threats. (2) Dual-Objective Optimization: Based on the dual objectives of safety and utility, the Utility Agent integrated within HarmonyGuard performs the Markovian real-time reasoning to evaluate the objectives and utilizes metacognitive capabilities for their optimization. Extensive evaluations on multiple benchmarks show that HarmonyGuard improves policy compliance by up to 38% and task completion by up to 20% over existing baselines, while achieving over 90% policy compliance across all tasks. Our project is available here: https://github.com/YurunChen/HarmonyGuard.