Visual Inception: Compromising Long-term Planning in Agentic Recommenders via Multimodal Memory Poisoning

arXiv 2604.16966 ↗First posted 18 Apr 2026cs.CR cs.AIcited by 4

On the map: LLM agents and tool use → Agent threats, guardrails and governance

Abstract (arXiv)

The evolution from static ranking models to Agentic Recommender Systems (Agentic RecSys) empowers AI agents to maintain long-term user profiles and autonomously plan service tasks. While this paradigm shift enhances personalization, it introduces a vulnerability: reliance on Long-term Memory (LTM). In this paper, we uncover a threat termed "Visual Inception." Unlike traditional adversarial attacks that seek immediate misclassification, Visual Inception injects triggers into user-uploaded images (e.g., lifestyle photos) that act as "sleeper agents" within the system's memory. When retrieved during future planning, these poisoned memories hijack the agent's reasoning chain, steering it toward adversary-defined goals (e.g., promoting high-margin products) without prompt injection. To mitigate this, we propose CognitiveGuard, a dual-process defense framework inspired by human cognition. It consists of a System 1 Perceptual Sanitizer (diffusion-based purification) to cleanse sensory inputs and a System 2 Reasoning Verifier (counterfactual consistency checks) to detect anomalies in memory-driven planning. Extensive experiments on a mock e-commerce agent environment demonstrate that Visual Inception achieves about 85% Goal-Hit Rate (GHR), while CognitiveGuard reduces this risk to around 10% with configurable latency trade-offs (about 1.5s in lite mode to about 6.5s for full sequential verification), without quality degradation under our setup.

Key facts. Cited by 4 papers in the map (data through 13 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 46 of its 59 references that are in the map. First posted on arXiv 18 Apr 2026.

Builds on

46 of 59 references.

Closest ancestors · 5

Same direction, newest first

Other references · 41

2024The Llama 3 Herd of Modelscited by 14,486
2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,107
2014Explaining and Harnessing Adversarial Examplescited by 7,336
2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,379
2017Billion-scale similarity search with GPUscited by 2,490
2023A Survey on Large Language Model based Autonomous Agentscited by 2,218
2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,170
2023MemGPT: Towards LLMs as Operating Systemscited by 1,354
2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,229
2024A Survey on the Memory Mechanism of Large Language Model based Agentscited by 625
2020On Adaptive Attacks to Adversarial Example Defensescited by 554
2018Black-box Adversarial Attacks with Limited Queries and Informationcited by 499

29 more

2018Spectral Signatures in Backdoor Attackscited by 481
2017Decision-Based Adversarial Attacks: Reliable Attacks Against Black-Box Machine Learning Modelscited by 460
2019STRIP: A Defence Against Trojan Attacks on Deep Neural Networkscited by 456
2023Causal Reasoning and Large Language Models: Opening a New Frontier for Causalitycited by 352
2023Jailbreak in pieces: Compositional Adversarial Attacks on Multi-Modal Language Modelscited by 272
2024Certifiably Robust RAG against Retrieval Corruptioncited by 149
2024RAGBench: Explainable Benchmark for Retrieval-Augmented Generation Systemscited by 116
2023BadCLIP: Dual-Embedding Guided Backdoor Attack on Multimodal Contrastive Learningcited by 114
2024Reducing hallucination in structured outputs via Retrieval-Augmented Generationcited by 93
2024Attention Tracker: Detecting Prompt Injection Attacks in LLMscited by 92
2023Ignore This Title and HackAPrompt: Exposing Systemic Vulnerabilities of LLMs through a Global Scale Prompt Hacking Competitioncited by 68
2022Evaluating the Adversarial Robustness of Adaptive Test-time Defensescited by 64
2024Phantom: General Backdoor Attacks on Retrieval Augmented Language Generationcited by 49
2025A Survey on LLM-powered Agents for Recommender Systemscited by 49
2023Faithful Explanations of Black-box NLP Models Using LLM-generated Counterfactualscited by 44
2024Immune: Improving Safety Against Jailbreaks in Multi-modal LLMs via Inference-Time Alignmentcited by 44
2024Defense Against Prompt Injection Attack by Leveraging Attack Techniquescited by 40
2024ADBM: Adversarial diffusion bridge model for reliable adversarial purificationcited by 24
2025MM-PoisonRAG: Disrupting Multimodal RAG with Local and Global Poisoning Attackscited by 24
2024RecAI: Leveraging Large Language Models for Next-Generation Recommender Systemscited by 20
2025FC-Attack: Jailbreaking Multimodal Large Language Models via Auto-Generated Flowchartscited by 17
2024Securing Vision-Language Models with a Robust Encoder Against Jailbreak and Adversarial Attackscited by 15
2024Adversarial Attacks to Multi-Modal Modelscited by 12
2024Adversarial Guided Diffusion Models for Adversarial Purificationcited by 8
2025One Pic is All it Takes: Poisoning Visual Document Retrieval Augmented Generation with a Single Imagecited by 8
2025NeuroGenPoisoning: Neuron-Guided Attacks on Retrieval-Augmented Generation of LLM via Genetic Optimization of External Knowledgecited by 7
2024DiffBreak: Is Diffusion-Based Purification Robust?cited by 6
2024Developing a Reliable, Fast, General-Purpose Hallucination Detection and Mitigation Servicecited by 2
2025CoDefend: Cross-Modal Collaborative Defense via Diffusion Purification and Prompt Optimizationcited by 2

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2026: 4.

4 papers.

Similar, not cited

Close in wording, no citation either way