# Enhancing Conversational Agents via Task-Oriented Adversarial Memory Adaptation

arXiv: https://arxiv.org/abs/2601.21797 · HTML: https://mlsanity.com/paper/2601.21797/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 42 references that are in the map.
- First posted on arXiv 29 Jan 2026.

## Builds on (most cited first, 24 of 24)

- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [A-MEM: Agentic Memory for LLM Agents](https://mlsanity.com/paper/2502.12110/) (2025)
- [Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory](https://mlsanity.com/paper/2504.19413/) (2025)
- [LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory](https://mlsanity.com/paper/2410.10813/) (2024)
- [A Survey on the Memory Mechanism of Large Language Model based Agents](https://mlsanity.com/paper/2404.13501/) (2024)
- [MovieChat: From Dense Token to Sparse Memory for Long Video Understanding](https://mlsanity.com/paper/2307.16449/) (2023)
- [Zep: A Temporal Knowledge Graph Architecture for Agent Memory](https://mlsanity.com/paper/2501.13956/) (2025)
- [Memory in the Age of AI Agents](https://mlsanity.com/paper/2512.13564/) (2025)
- [A Human-Inspired Reading Agent with Gist Memory of Very Long Contexts](https://mlsanity.com/paper/2402.09727/) (2024)
- [Recent Advances in Deep Learning Based Dialogue Systems: A Systematic Survey](https://mlsanity.com/paper/2105.04387/) (2021)
- [From Human Memory to AI Memory: A Survey on Memory Mechanisms in the Era of LLMs](https://mlsanity.com/paper/2504.15965/) (2025)
- [Large Language Models Can Learn Temporal Reasoning](https://mlsanity.com/paper/2401.06853/) (2024)
- [On Memory Construction and Retrieval for Personalized Conversational Agents](https://mlsanity.com/paper/2502.05589/) (2025)
- [A Survey on Proactive Dialogue Systems: Problems, Methods, and Prospects](https://mlsanity.com/paper/2305.02750/) (2023)
- [Hierarchical Memory for High-Efficiency Long-Term Reasoning in LLM Agents](https://mlsanity.com/paper/2507.22925/) (2025)
- [What Deserves Memory: Adaptive Memory Distillation for LLM Agents](https://mlsanity.com/paper/2508.03341/) (2025)
- [O-Mem: Omni Memory System for Personalized, Long Horizon, Self-Evolving Agents](https://mlsanity.com/paper/2511.13593/) (2025)
- [AgentCF++: Memory-enhanced LLM-based Agents for Popularity-aware Cross-domain Recommendations](https://mlsanity.com/paper/2502.13843/) (2025)
- [AssoMem: Scalable Memory QA with Multi-Signal Associative Retrieval](https://mlsanity.com/paper/2510.10397/) (2025)
- [Memory-QA: Answering Recall Questions Based on Multimodal Memories](https://mlsanity.com/paper/2509.18436/) (2025)
- [Encode-Store-Retrieve: Augmenting Human Memory through Language-Encoded Egocentric Perception](https://mlsanity.com/paper/2308.05822/) (2023)
- [Query Understanding in LLM-based Conversational Information Seeking](https://mlsanity.com/paper/2504.06356/) (2025)
- [Search-Based Interaction For Conversation Recommendation via Generative Reward Model Based Simulated User](https://mlsanity.com/paper/2504.20458/) (2025)
- [Enhancing the Patent Matching Capability of Large Language Models via the Memory Graph](https://mlsanity.com/paper/2504.14845/) (2025)

## Built on by (most cited first, 1 of 1)

- [MemSearch-o1: Empowering Large Language Models with Reasoning-Aligned Memory Growth in Agentic Search](https://mlsanity.com/paper/2604.17265/) (2026)

## Abstract (arXiv)

Conversational agents struggle to handle long conversations due to context window limitations. Therefore, memory systems are developed to leverage essential historical information. Existing memory systems typically follow a pipeline of offline memory construction and update, and online retrieval. Despite the flexible online phase, the offline phase remains fixed and task-independent. In this phase, memory construction operates under a predefined workflow and fails to emphasize task relevant information. Meanwhile, memory updates are guided by generic metrics rather than task specific supervision. This leads to a misalignment between offline memory preparation and task requirements, which undermines downstream task performance. To this end, we propose an Adversarial Memory Adaptation mechanism (AMA) that aligns memory construction and update with task objectives by simulating task execution. Specifically, first, a challenger agent generates question answer pairs based on the original dialogues. The constructed memory is then used to answer these questions, simulating downstream inference. Subsequently, an evaluator agent assesses the responses and performs error analysis. Finally, an adapter agent analyzes the error cases and performs dual level updates on both the construction strategy and the content. Through this process, the memory system receives task aware supervision signals in advance during the offline phase, enhancing its adaptability to downstream tasks. AMA can be integrated into various existing memory systems, and extensive experiments on long dialogue benchmark LoCoMo demonstrate its effectiveness.