# When Routine Chats Turn Toxic: Unintended Long-Term State Poisoning in Personalized Agents

arXiv: https://arxiv.org/abs/2605.06731 · HTML: https://mlsanity.com/paper/2605.06731/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use.
- Builds on 35 of its 37 references that are in the map.
- First posted on arXiv 7 May 2026.

## Placed in

- LLM agents and tool use / [Agent threats, guardrails and governance](https://mlsanity.com/agents/agent-threats-guardrails-and-governance/)

## Builds on (most cited first, 30 of 35)

- [Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection](https://mlsanity.com/paper/2302.12173/) (2023)
- [OpenHands: An Open Platform for AI Software Developers as Generalist Agents](https://mlsanity.com/paper/2407.16741/) (2024)
- [DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models](https://mlsanity.com/paper/2512.02556/) (2025)
- [WildChat: 1M ChatGPT Interaction Logs in the Wild](https://mlsanity.com/paper/2405.01470/) (2024)
- [LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Dataset](https://mlsanity.com/paper/2309.11998/) (2023)
- [AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents](https://mlsanity.com/paper/2410.09024/) (2024)
- [Detecting Language Model Attacks with Perplexity](https://mlsanity.com/paper/2308.14132/) (2023)
- [AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Bases](https://mlsanity.com/paper/2407.12784/) (2024)
- [Agent Security Bench (ASB): Formalizing and Benchmarking Attacks and Defenses in LLM-based Agents](https://mlsanity.com/paper/2410.02644/) (2024)
- [Justice or Prejudice? Quantifying Biases in LLM-as-a-Judge](https://mlsanity.com/paper/2410.02736/) (2024)
- [Formalizing and Benchmarking Prompt Injection Attacks and Defenses](https://mlsanity.com/paper/2310.12815/) (2023)
- [Humans or LLMs as the Judge? A Study on Judgement Biases](https://mlsanity.com/paper/2402.10669/) (2024)
- [Self-Preference Bias in LLM-as-a-Judge](https://mlsanity.com/paper/2410.21819/) (2024)
- [AgentSpec: Customizable Runtime Enforcement for Safe and Reliable LLM Agents](https://mlsanity.com/paper/2503.18666/) (2025)
- [Safety at Scale: A Comprehensive Survey of Large Model and Agent Safety](https://mlsanity.com/paper/2502.05206/) (2025)
- [Claw-Eval: Towards Trustworthy Evaluation of Autonomous Agents](https://mlsanity.com/paper/2604.06132/) (2026)
- [Caution for the Environment: Multimodal LLM Agents are Susceptible to Environmental Distractions](https://mlsanity.com/paper/2408.02544/) (2024)
- [VeriGuard: Enhancing LLM Agent Safety via Verified Code Generation](https://mlsanity.com/paper/2510.05156/) (2025)
- [Zombie Agents: Persistent Control of Self-Evolving LLM Agents via Self-Reinforcing Injections](https://mlsanity.com/paper/2602.15654/) (2026)
- [From Assistant to Double Agent: Formalizing and Benchmarking Attacks on OpenClaw for Personalized Local AI Agent](https://mlsanity.com/paper/2602.08412/) (2026)
- [AgentSentry: Mitigating Indirect Prompt Injection in LLM Agents via Temporal Causal Diagnostics and Context Purification](https://mlsanity.com/paper/2602.22724/) (2026)
- [Taming OpenClaw: Security Analysis and Mitigation of Autonomous LLM Agent Threats](https://mlsanity.com/paper/2603.11619/) (2026)
- [ProbGuard: Proactive Runtime Monitoring for LLM Agent Safety via Probabilistic Prediction](https://mlsanity.com/paper/2508.00500/) (2025)
- [SkillTrojan: Backdoor Attacks on Skill-Based Agent Systems](https://mlsanity.com/paper/2604.06811/) (2026)
- [Your Agent, Their Asset: A Real-World Safety Analysis of OpenClaw](https://mlsanity.com/paper/2604.04759/) (2026)
- [From Storage to Steering: Memory Control Flow Attacks on LLM Agents](https://mlsanity.com/paper/2603.15125/) (2026)
- [AgentWorm: Self-Propagating Attacks Across LLM Agent Ecosystems](https://mlsanity.com/paper/2603.15727/) (2026)
- [ClawSafety: "Safe" LLMs, Unsafe Agents](https://mlsanity.com/paper/2604.01438/) (2026)
- [TraceSafe: A Systematic Assessment of LLM Guardrails on Multi-Step Tool-Calling Trajectories](https://mlsanity.com/paper/2604.07223/) (2026)
- [When Benign Inputs Lead to Severe Harms: Eliciting Unsafe Unintended Behaviors of Computer-Use Agents](https://mlsanity.com/paper/2602.08235/) (2026)

## Built on by (most cited first, 1 of 1)

- [SINGED: Correct Outputs Do Not Certify Safe Execution in LLM Agents](https://mlsanity.com/paper/2609.35889/) (2026)

## Abstract (arXiv)

Personalized LLM agents maintain persistent cross-session state to support long-horizon collaboration. Yet, this persistence introduces a subtle but critical security vulnerability: routine user-agent interactions can gradually reshape an agent's long-term state, inadvertently weakening future confirmation boundaries, expanding tool-use defaults, and escalating autonomous behavior over time. We formalize this risk as unintended long-term state poisoning. To systematically study it, we introduce the Unintended Long-Term State Poisoning Bench (ULSPB), a bilingual benchmark comprising 350 settings spanning five assistance categories, seven interaction patterns, 24-turn routine interactions, and matched single-injection counterparts. Furthermore, we define the Harm Score (HS), a state-centric metric that quantifies authorization drift, tool-use escalation, and unchecked autonomy. Experiments on OpenClaw with four backbone LLMs demonstrate that, while single-injection is generally effective, routine conversations alone can substantially poison long-term state, primarily corrupting memory-centric artifacts. Evaluations seeded with real-world user interactions confirm that this risk is not a mere artifact of synthetic prompts. To mitigate this threat, we propose StateGuard, a lightweight, post-execution defense that audits state diffs at the writeback boundary and selectively rolls back dangerous edits. Across all evaluated models, StateGuard reduces HS to near zero and lowers false-negative rates, with acceptable high false-positive rates under a safety-first writeback defense and minimal overhead.