# Invisible Ink Threats: Adversarial Goals Behind Legitimate Tasks in Computer-Use Agents

arXiv: https://arxiv.org/abs/2608.02018 · HTML: https://mlsanity.com/paper/2608.02018/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use.
- Builds on 18 of its 34 references that are in the map.
- First posted on arXiv 3 Aug 2026.

## Placed in

- LLM agents and tool use / [Agent threats, guardrails and governance](https://mlsanity.com/agents/agent-threats-guardrails-and-governance/)
- LLM safety and security / [Prompt injection and agent security](https://mlsanity.com/safety/prompt-injection-and-agent-security/)

## Builds on (most cited first, 18 of 18)

- [WebArena: A Realistic Web Environment for Building Autonomous Agents](https://mlsanity.com/paper/2307.13854/) (2023)
- [Mind2Web: Towards a Generalist Agent for the Web](https://mlsanity.com/paper/2306.06070/) (2023)
- [OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments](https://mlsanity.com/paper/2404.07972/) (2024)
- [Identifying the Risks of LM Agents with an LM-Emulated Sandbox](https://mlsanity.com/paper/2309.15817/) (2023)
- [TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Tasks](https://mlsanity.com/paper/2412.14161/) (2024)
- [Mapping Natural Language Instructions to Mobile UI Action Sequences](https://mlsanity.com/paper/2005.03776/) (2020)
- [Attacking Vision-Language Computer Agents via Pop-ups](https://mlsanity.com/paper/2411.02391/) (2024)
- [OS-Harm: A Benchmark for Measuring Safety of Computer Use Agents](https://mlsanity.com/paper/2506.14866/) (2025)
- [Can LLMs Separate Instructions From Data? And What Do We Even Mean By That?](https://mlsanity.com/paper/2403.06833/) (2024)
- [The Obvious Invisible Threat: LLM-Powered GUI Agents' Vulnerability to Fine-Print Injections](https://mlsanity.com/paper/2504.11281/) (2025)
- [Commercial LLM Agents Are Already Vulnerable to Simple Yet Dangerous Attacks](https://mlsanity.com/paper/2502.08586/) (2025)
- [RedTeamCUA: Realistic Adversarial Testing of Computer-Use Agents in Hybrid Web-OS Environments](https://mlsanity.com/paper/2505.21936/) (2025)
- [GLM-5V-Turbo: Toward a Native Foundation Model for Multimodal Agents](https://mlsanity.com/paper/2604.26752/) (2026)
- [DoomArena: A framework for Testing AI Agents Against Evolving Security Threats](https://mlsanity.com/paper/2504.14064/) (2025)
- [SecureWebArena: A Holistic Security Evaluation Benchmark for LVLM-based Web Agents](https://mlsanity.com/paper/2510.10073/) (2025)
- [Parameter-Efficient Fine-Tuning of Large Language Models via Deconvolution in Subspace](https://mlsanity.com/paper/2503.01419/) (2025)
- [Preference Redirection via Attention Concentration: An Attack on Computer Use Agents](https://mlsanity.com/paper/2604.08005/) (2026)
- [Online Deep Learning from Doubly-Streaming Data](https://mlsanity.com/paper/2204.11793/) (2022)

## Built on by (most cited first, 1 of 1)

- [ERPBench: A State-Grounded Evaluation Paradigm for Computer-Use Agents in Enterprise Software](https://mlsanity.com/paper/2609.17885/) (2026)

## Abstract (arXiv)

Computer-use agents (CUAs), which empower large language models to autonomously operate operating systems and the web, are increasingly vulnerable to indirect prompt injection attacks. A widely adopted defense is the human-in-the-loop paradigm, in which the agent pauses for explicit user confirmation before executing sensitive operations. While effective against conspicuously high-harm attacks, this defense offers little protection against what we term Invisible Ink Threats: low-harm injected goals, such as starring a repository or installing a package, that are behaviorally indistinguishable from legitimate task execution and thus evade both model safety mechanisms and human oversight. To systematically investigate this blind spot, we present II-Bench, a collection of seemingly harmless adversarial tasks. II-Bench comprises 444 examples targeting confidentiality and integrity attacks across three platforms, spanning three attack categories: page navigation and interaction, sensitive information exfiltration, and code download and execution. Each category is instantiated in both natural language and code forms under two levels of instruction specificity. Furthermore, we construct HITLCUA, a comprehensive adversarial testing framework that integrates a real virtual machine operating system environment with isolated Docker-based web platforms, and simulates human participation by allowing CUAs to consult an API-simulated user before proceeding with suspicious operations. Extensive evaluations of leading CUAs reveal that low-harm injections frequently bypass both agent defenses and simulated user review, exposing severe and previously underexplored security risks in current CUAs.