mlsanityData to 7 Oct 2026

Real AI Agents with Fake Memories: Fatal Context Manipulation Attacks on Web3 Agents

arXiv 2503.16248 ↗First posted 20 Mar 2025cs.CR cs.AIcited by 23

On the map: LLM safety and security → Prompt injection and agent security

Abstract (arXiv)

AI agents integrated with Web3 offer autonomy and openness but raise security concerns as they interact with financial protocols and immutable smart contracts. This paper investigates the vulnerabilities of AI agents within blockchain-based financial ecosystems when exposed to adversarial threats in real-world scenarios. We introduce the concept of context manipulation -- a comprehensive attack vector that exploits unprotected context surfaces, including input channels, memory modules, and external data feeds. It expands on traditional prompt injection and reveals a more stealthy and persistent threat: memory injection. Using ElizaOS, a representative decentralized AI agent framework for automated Web3 operations, we showcase that malicious injections into prompts or historical records can trigger unauthorized asset transfers and protocol violations which could be financially devastating in reality. To quantify these risks, we introduce CrAIBench, a Web3-focused benchmark covering 150+ realistic blockchain tasks. such as token transfers, trading, bridges, and cross-chain interactions, and 500+ attack test cases using context manipulation. Our evaluation results confirm that AI models are significantly more vulnerable to memory injection compared to prompt injection. Finally, we evaluate a comprehensive defense roadmap, finding that prompt-injection defenses and detectors only provide limited protection when stored context is corrupted, whereas fine-tuning-based defenses substantially reduce attack success rates while preserving performance on single-step tasks. These results underscore the urgent need for AI agents that are both secure and fiduciarily responsible in blockchain environments.

Key facts. Cited by 23 papers in the map (data through 7 Oct 2026). Placed in Prompt injection and agent security, one of 12 directions in LLM safety and security. Builds on 32 of its 74 references that are in the map. First posted on arXiv 20 Mar 2025.

Builds on

32 of 74 references.

Closest ancestors · 10

Same direction, newest first

  1. 2024AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Bases17 Jul
  2. 2024AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents19 Jun
  3. 2024Defending Against Indirect Prompt Injection Attacks With Spotlighting20 Mar
  4. 2024Neural Exec: Learning (and Learning from) Execution Triggers for Prompt Injection Attacks6 Mar
  5. 2024InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents5 Mar
  6. 2024Watch Out for Your Agents! Investigating Backdoor Threats to LLM-Based Agents17 Feb
  7. 2024StruQ: Defending Against Prompt Injection with Structured Queries9 Feb
  8. 2023Benchmarking and Defending Against Indirect Prompt Injection Attacks on Large Language Models21 Dec
  9. 2023Prompt Injection attack against LLM-integrated Applications8 Jun
  10. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection23 Feb

Other references · 22

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  3. 2023Mistral 7Bcited by 3,065
  4. 2022Language Models (Mostly) Know What They Knowcited by 1,783
  5. 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,280
  6. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,124
  7. 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 760
  8. 2023Identifying the Risks of LM Agents with an LM-Emulated Sandboxcited by 514
  9. 2024Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Trainingcited by 478
  10. 2024WebVoyager: Building an End-to-End Web Agent with Large Multimodal Modelscited by 424
  11. 2024PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Modelscited by 251
  12. 2021Adversarial GLUE: A Multi-Task Benchmark for Robustness Evaluation of Language Modelscited by 227
10 more
  1. 2024APIGen: Automated Pipeline for Generating Verifiable and Diverse Function-Calling Datasetscited by 188
  2. 2024Certifiably Robust RAG against Retrieval Corruptioncited by 144
  3. 2025Memory Injection Attacks on LLM Agents via Query-Only Interactioncited by 127
  4. 2024BadChain: Backdoor Chain-of-Thought Prompting for Large Language Modelscited by 74
  5. 2023LLM Platform Security: Applying a Systematic Evaluation Framework to OpenAI's ChatGPT Pluginscited by 49
  6. 2025Commercial LLM Agents Are Already Vulnerable to Simple Yet Dangerous Attackscited by 49
  7. 2025Demonstrating specification gaming in reasoning modelscited by 48
  8. 2024Breaking ReAct Agents: Foot-in-the-Door Attack Will Get You Incited by 21
  9. 2025Eliza: A Web3 friendly AI Agent Operating Systemcited by 14
  10. 2024Self-Evaluation as a Defense Against Adversarial Attacks on LLMscited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 10; 2026: 13.

23 papers.

Sort:

By area:

  1. 2025From Prompt Injections to Protocol Exploits: Threats in LLM-Powered AI Agents Workflowscited by 57
  2. 2026A Survey of Agent Memory in the Second Half: Towards Self-Evolving and Long-Horizon Agentscited by 40
  3. 2025A Survey on the Safety and Security Threats of Computer-Using Agents: JARVIS or Ultron?cited by 23
  4. 2026Poison Once, Exploit Forever: Environment-Injected Memory Poisoning Attacks on Web Agentscited by 18
  5. 2025Trustless Autonomy: Understanding Motivations, Benefits, and Governance Dilemmas in Self-Sovereign Decentralized AI Agentscited by 10
  6. 2025Context manipulation attacks : Web agents are susceptible to corrupted memorycited by 10
  7. 2026SoK: Security of Autonomous LLM Agents in Agentic Commercecited by 8
  8. 2025Spore in the Wild: A Case Study of Spore.fun as an Open-Environment Evolution Experiment with Sovereign AI Agents on TEE-Secured Blockchainscited by 5
  9. 2026Toward Secure LLM Agents: Threat Surfaces, Attacks, Defenses, and Evaluationcited by 5
  10. 2025Red-Teaming Coding Agents from a Tool-Invocation Perspective: An Empirical Security Assessmentcited by 4
  11. 2025Giving AI Agents Access to Cryptocurrency and Smart Contracts Creates New Vectors of AI Harmcited by 3
  12. 2025SoK: Security and Privacy of AI Agents for Blockchaincited by 3
11 more
  1. 2025Hide-and-Shill: A Reinforcement Learning Framework for Market Manipulation Detection in Symphony-a Decentralized Multi-Agent Systemcited by 2
  2. 2026Visual Memory Injection Attacks for Multi-Turn Conversationscited by 2
  3. 2026Grounded Continuation: A Linear-Time Runtime Verifier for LLM Conversationscited by 2
  4. 2026When Claws Remember but Do Not Tell: Stealthy Memory Injection in Persistent Personal Agentscited by 2
  5. 2025Epistemic Bias Injection: Manipulating LLM Opinion via Selective Context Retrievalcited by 1
  6. 2026The Containment Gap: How Deployed Agentic AI Frameworks Fail Public-Facing Safety Requirementscited by 1
  7. 2026Proof-of-Execution Memory: Defending LLM Agents Against Forged-Reasoning Attacks by Verifying What Actually Happenedcited by 1
  8. 2026MAFIA: Query-Only Memory Attacks via Probing and Factual Injection against Audited LLM Agentscited by 0
  9. 2026On Understanding, Identifying, and Mitigating Vulnerabilities in Agentic Large Language Modelscited by 0
  10. 2026Rethinking Agent Security as a Networking Problemcited by 0
  11. 2026From Attack Success to Attack Severity: Counterfactual Memory Attacks on LLM Agentscited by 0