# MemoPhishAgent: Memory-Augmented Multi-Modal LLM Agent for Phishing URL Detection

arXiv: https://arxiv.org/abs/2602.21394 · HTML: https://mlsanity.com/paper/2602.21394/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use.
- Builds on 16 of its 49 references that are in the map.
- First posted on arXiv 24 Feb 2026.

## Placed in

- LLM agents and tool use / [Agent threats, guardrails and governance](https://mlsanity.com/agents/agent-threats-guardrails-and-governance/)

## Builds on (most cited first, 16 of 16)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [REALM: Retrieval-Augmented Language Model Pre-Training](https://mlsanity.com/paper/2002.08909/) (2020)
- [A-MEM: Agentic Memory for LLM Agents](https://mlsanity.com/paper/2502.12110/) (2025)
- [Tranco: A Research-Oriented Top Sites Ranking Hardened Against Manipulation](https://mlsanity.com/paper/1806.01156/) (2018)
- [KnowPhish: Large Language Models Meet Multimodal Knowledge Graphs for Enhancing Reference-Based Phishing Detection](https://mlsanity.com/paper/2403.02253/) (2024)
- [RL-JACK: Reinforcement Learning-powered Black-box Jailbreaking Attack against LLMs](https://mlsanity.com/paper/2406.08725/) (2024)
- [ParaFuzz: An Interpretability-Driven Technique for Detecting Poisoned Samples in NLP](https://mlsanity.com/paper/2308.02122/) (2023)
- [PhishAgent: A Robust Multimodal Agent for Phishing Webpage Detection](https://mlsanity.com/paper/2408.10738/) (2024)
- [URLTran: Improving Phishing URL Detection Using Transformers](https://mlsanity.com/paper/2106.05256/) (2021)
- [TAI3: Testing Agent Integrity in Interpreting User Intent](https://mlsanity.com/paper/2506.07524/) (2025)
- [Evaluating the Effectiveness and Robustness of Visual Similarity-based Phishing Detection Models](https://mlsanity.com/paper/2405.19598/) (2024)
- [AssistRAG: Boosting the Potential of Large Language Models with an Intelligent Information Assistant](https://mlsanity.com/paper/2411.06805/) (2024)
- [Automated Phishing Detection Using URLs and Webpages](https://mlsanity.com/paper/2408.01667/) (2024)
- [PhishIntel: Toward Practical Deployment of Reference-Based Phishing Detection](https://mlsanity.com/paper/2412.09057/) (2024)
- [Phishing Webpage Detection: Unveiling the Threat Landscape and Investigating Detection Techniques](https://mlsanity.com/paper/2509.08424/) (2025)

## Built on by (most cited first, 1 of 1)

- [The Fragility of Trigger-Tag Mechanisms for Misuse Detection in Open-Weight LLMs](https://mlsanity.com/paper/2610.03124/) (2026)

## Abstract (arXiv)

Traditional phishing website detection relies on static heuristics or reference lists, which lag behind rapidly evolving attacks. While recent systems incorporate large language models (LLMs), they are still prompt-based, deterministic pipelines that underutilize reasoning capability. We present MemoPhishAgent (MPA), a memory-augmented multi-modal LLM agent that dynamically orchestrates phishing-specific tools and leverages episodic memories of past reasoning trajectories to guide decisions on recurring and novel threats. On two public datasets, MPA outperforms three state-of-the-art (SOTA) baselines, improving recall by 13.6%. To better reflect realistic, user-facing phishing detection performance, we further evaluate MPA on a benchmark of real-world suspicious URLs actively crawled from five social media platforms, where it improves recall by 20%. Detailed analysis shows episodic memory contributes up to 27% recall gain without introducing additional computational overhead. The ablation study confirms the necessity of the agent-based approach compared to prompt-based baselines and validates the effectiveness of our tool design. Finally, MPA is deployed in production, processing 60K targeted high-risk URLs weekly, and achieving 91.44% recall, providing proactive protection for millions of customers. Together, our results show that combining multi-modal reasoning with episodic memory yields robust phishing detection in realistic user-exposure settings. Our implementation is available at https://github.com/XuanChen-xc/MemoPhishAgent.git.