# Personalized Large Language Model Assistant with Evolving Conditional Memory

arXiv: https://arxiv.org/abs/2312.17257 · HTML: https://mlsanity.com/paper/2312.17257/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 34 references that are in the map.
- First posted on arXiv 22 Dec 2023.

## Builds on (most cited first, 24 of 24)

- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [Self-Instruct: Aligning Language Models with Self-Generated Instructions](https://mlsanity.com/paper/2212.10560/) (2022)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [Large Language Models are not Fair Evaluators](https://mlsanity.com/paper/2305.17926/) (2023)
- [In-Context Retrieval-Augmented Language Models](https://mlsanity.com/paper/2302.00083/) (2023)
- [Wizard of Wikipedia: Knowledge-Powered Conversational agents](https://mlsanity.com/paper/1811.01241/) (2018)
- [Beyond Goldfish Memory: Long-Term Open-Domain Conversation](https://mlsanity.com/paper/2107.07567/) (2021)
- [Internet-Augmented Dialogue Generation](https://mlsanity.com/paper/2107.07566/) (2021)
- [Rethinking with Retrieval: Faithful Large Language Model Inference](https://mlsanity.com/paper/2301.00303/) (2022)
- [Open-Retrieval Conversational Question Answering](https://mlsanity.com/paper/2005.11364/) (2020)
- [TopiOCQA: Open-domain Conversational Question Answering with Topic Switching](https://mlsanity.com/paper/2110.00768/) (2021)
- [Question Rewriting for Conversational Question Answering](https://mlsanity.com/paper/2004.14652/) (2020)
- [Learning to Select Knowledge for Response Generation in Dialog Systems](https://mlsanity.com/paper/1902.04911/) (2019)
- [BERT with History Answer Embedding for Conversational Question Answering](https://mlsanity.com/paper/1905.05412/) (2019)
- [Skeleton-to-Response: Dialogue Generation Guided by Retrieval Memory](https://mlsanity.com/paper/1809.05296/) (2018)
- [A Controllable Model of Grounded Response Generation](https://mlsanity.com/paper/2005.00613/) (2020)
- [Look before you Hop: Conversational Question Answering over Knowledge Graphs Using Judicious Context Expansion](https://mlsanity.com/paper/1910.03262/) (2019)
- [Response Generation by Context-aware Prototype Editing](https://mlsanity.com/paper/1806.07042/) (2018)
- [RETA-LLM: A Retrieval-Augmented Large Language Model Toolkit](https://mlsanity.com/paper/2306.05212/) (2023)
- [Hindsight: Posterior-guided training of retrievers for improved open-ended generation](https://mlsanity.com/paper/2110.07752/) (2021)
- [Conversing by Reading: Contentful Neural Conversation with On-demand Machine Reading](https://mlsanity.com/paper/1906.02738/) (2019)
- [Controlling Dialogue Generation with Semantic Exemplars](https://mlsanity.com/paper/2008.09075/) (2020)
- [A Hybrid Retrieval-Generation Neural Conversation Model](https://mlsanity.com/paper/1904.09068/) (2019)

## Built on by (most cited first, 25 of 25)

- [HiAgent: Hierarchical Working Memory Management for Solving Long-Horizon Agent Tasks with Large Language Model](https://mlsanity.com/paper/2408.09559/) (2024)
- [From Recall to Forgetting: Benchmarking Long-Term Memory for Personalized Agents](https://mlsanity.com/paper/2604.20006/) (2026)
- [From Single to Multi-Granularity: Toward Long-Term Memory Association and Selection of Conversational Agents](https://mlsanity.com/paper/2505.19549/) (2025)
- [A Survey on Personalized and Pluralistic Preference Alignment in Large Language Models](https://mlsanity.com/paper/2504.07070/) (2025)
- [From Storage to Experience: A Survey on the Evolution of LLM Agent Memory Mechanisms](https://mlsanity.com/paper/2605.06716/) (2026)
- [Pre-Storage Reasoning for Episodic Memory: Shifting Inference Burden to Memory for Personalized Dialogue](https://mlsanity.com/paper/2509.10852/) (2025)
- [Enabling Personalized Long-term Interactions in LLM-based Agents through Persistent Memory and User Profiles](https://mlsanity.com/paper/2510.07925/) (2025)
- [DeepPersona: A Generative Engine for Scaling Deep Synthetic Personas](https://mlsanity.com/paper/2511.07338/) (2025)
- [Toward Personalized LLM-Powered Agents: Foundations, Evaluation, and Future Directions](https://mlsanity.com/paper/2602.22680/) (2026)
- [Mem-PAL: Towards Memory-based Personalized Dialogue Assistants for Long-term User-Agent Interaction](https://mlsanity.com/paper/2511.13410/) (2025)
- [LifeSim: Long-Horizon User Life Simulator for Personalized Assistant Evaluation](https://mlsanity.com/paper/2603.12152/) (2026)
- [MemORAI: Memory Organization and Retrieval via Adaptive Graph Intelligence for LLM Conversational Agents](https://mlsanity.com/paper/2605.01386/) (2026)
- [PEToolLLM: Towards Personalized Tool Learning in Large Language Models](https://mlsanity.com/paper/2502.18980/) (2025)
- [DaMo: Data Mixing Optimizer in Fine-tuning Multimodal LLMs for Mobile Phone Agents](https://mlsanity.com/paper/2510.19336/) (2025)
- [DYCP: Dynamic Context Pruning for Long-Form Dialogue with LLMs](https://mlsanity.com/paper/2601.07994/) (2026)
- [Language Models Don't Know What You Want: Evaluating Personalization in Deep Research Needs Real Users](https://mlsanity.com/paper/2603.16120/) (2026)
- [Opal: Private Memory for Personal AI](https://mlsanity.com/paper/2604.02522/) (2026)
- [Memory Is Not Always Needed: Characterizing Conditional Memory in Scientific Reasoning](https://mlsanity.com/paper/2608.23982/) (2026)
- [Back to Basics: Let Conversational Agents Remember with Just Retrieval and Generation](https://mlsanity.com/paper/2604.11628/) (2026)
- [Hierarchical Graph Memory for LLM Agents with Path-level Localization and Rewrite](https://mlsanity.com/paper/2608.05095/) (2026)
- [Inferring Hidden User Models from the Behavior of Personalized LLM Agents](https://mlsanity.com/paper/2609.03815/) (2026)
- [Graph-Based Personalized Memory for LLM Agents: Representation, Evolution, Retrieval, and Evaluation](https://mlsanity.com/paper/2609.08599/) (2026)
- [AIM: A Privacy-Aware Interoperable Memory Framework for Multi-Agent Multi-User LLM Systems](https://mlsanity.com/paper/2609.12320/) (2026)
- [Privacy in Personalized AI Is a System Property, Not Just a Model Property](https://mlsanity.com/paper/2609.38289/) (2026)
- [BRACE: Differential Privacy for Dense Associative Memory with LSR Energy](https://mlsanity.com/paper/2610.11218/) (2026)

## Abstract (arXiv)

With the rapid development of large language models, AI assistants like ChatGPT have become increasingly integrated into people's works and lives but are limited in personalized services. In this paper, we present a plug-and-play framework that could facilitate personalized large language model assistants with evolving conditional memory. The personalized assistant focuses on intelligently preserving the knowledge and experience from the history dialogue with the user, which can be applied to future tailored responses that better align with the user's preferences. Generally, the assistant generates a set of records from the dialogue dialogue, stores them in a memory bank, and retrieves related memory to improve the quality of the response. For the crucial memory design, we explore different ways of constructing the memory and propose a new memorizing mechanism named conditional memory. We also investigate the retrieval and usage of memory in the generation process. We build the first benchmark to evaluate personalized assistants' ability from three aspects. The experimental results illustrate the effectiveness of our method.