# How Can Recommendation Feedback Evolve Agent Memory?

arXiv: https://arxiv.org/abs/2609.37544 · HTML: https://mlsanity.com/paper/2609.37544/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 30 of its 34 references that are in the map.
- First posted on arXiv 29 Sep 2026.

## Builds on (most cited first, 30 of 30)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [Let's Verify Step by Step](https://mlsanity.com/paper/2305.20050/) (2023)
- [Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection](https://mlsanity.com/paper/2310.11511/) (2023)
- [ExpeL: LLM Agents Are Experiential Learners](https://mlsanity.com/paper/2308.10144/) (2023)
- [Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory](https://mlsanity.com/paper/2504.19413/) (2025)
- [From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judge](https://mlsanity.com/paper/2411.16594/) (2024)
- [Absolute Zero: Reinforced Self-play Reasoning with Zero Data](https://mlsanity.com/paper/2505.03335/) (2025)
- [Dynamic Cheatsheet: Test-Time Learning with Adaptive Memory](https://mlsanity.com/paper/2504.07952/) (2025)
- [Evo-Memory: Benchmarking LLM Agent Test-time Learning with Self-Evolving Memory](https://mlsanity.com/paper/2511.20857/) (2025)
- [Memp: Exploring Agent Procedural Memory](https://mlsanity.com/paper/2508.06433/) (2025)
- [Self-Adapting Language Models](https://mlsanity.com/paper/2506.10943/) (2025)
- [AgentRefine: Enhancing Agent Generalization through Refinement Tuning](https://mlsanity.com/paper/2501.01702/) (2025)
- [PosterCraft: Rethinking High-Quality Aesthetic Poster Generation in a Unified Framework](https://mlsanity.com/paper/2506.10741/) (2025)
- [Learning Personalized Agents from Human Feedback](https://mlsanity.com/paper/2602.16173/) (2026)
- [CREATER: CTR-driven Advertising Text Generation with Controlled Pre-Training and Contrastive Fine-Tuning](https://mlsanity.com/paper/2205.08943/) (2022)
- [MemoryCD: Benchmarking Long-Context User Memory of LLM Agents for Lifelong Cross-Domain Personalization](https://mlsanity.com/paper/2603.25973/) (2026)
- [EvoMemBench: Benchmarking Agent Memory from a Self-Evolving Perspective](https://mlsanity.com/paper/2605.18421/) (2026)
- [Self-Evolved Reward Learning for LLMs](https://mlsanity.com/paper/2411.00418/) (2024)
- [Dynamic and Generalizable Process Reward Modeling](https://mlsanity.com/paper/2507.17849/) (2025)
- [CTR-Driven Advertising Image Generation with Multimodal Large Language Models](https://mlsanity.com/paper/2502.06823/) (2025)
- [CTR-Guided Generative Query Suggestion in Conversational Search](https://mlsanity.com/paper/2507.04072/) (2025)
- [PosterOmni: Generalized Artistic Poster Creation via Task Distillation and Unified Reward Feedback](https://mlsanity.com/paper/2602.12127/) (2026)
- [CTR-Driven Ad Text Generation via Online Feedback Preference Optimization](https://mlsanity.com/paper/2507.20227/) (2025)
- [Generative Explore-Exploit: Training-free Optimization of Generative Recommender Systems using LLM Optimizers](https://mlsanity.com/paper/2406.05255/) (2024)
- [PosterReward: Unlocking Accurate Evaluation for High-Quality Graphic Design Generation](https://mlsanity.com/paper/2603.29855/) (2026)
- [A Better Spur Should Start From Each Objective](https://mlsanity.com/paper/2609.08211/) (2026)

## Abstract (arXiv)

Content-generation agents continuously receive impressions, clicks, conversions, and negative feedback from recommendation systems, providing real-world outcome signals for memory evolution. However, these signals are delayed and noisy, confounded by audience composition, placement, and recommendation policies, and may result from the combined influence of multiple memories, making accurate attribution difficult. Existing methods rely primarily on immediate feedback or semantic retrieval and therefore struggle to reliably translate recommendation outcomes into memory fitness. To address this challenge, we propose TIDE (Trajectory-Informed Directed Memory Evolution), an external memory evolution framework driven by delayed recommendation feedback. We further introduce Memory Evolution Gain (MEG), which measures the utility improvement of evolved memory over a no memory baseline on strictly future tasks. TIDE treats memory as a capacity-constrained population of experiences: temporal and semantic credit assignment estimates contextual fitness, while responsibility credit distributes outcome signals according to the memories referenced during generation. These signals are then used to reinforce, crossover, mutate, or evict memories. On an e-commerce membership marketing content-generation agent, TIDE achieves a +7.75-percentage-point MEG in offline temporal replay and significantly improves both unique click-through rate (UCTR) and activation rate in an online A/B test. On a delayed-label benchmark, TIDE achieves the lowest mean absolute error (MAE) and root mean squared error (RMSE) and the highest MEG among the compared methods, demonstrating its effectiveness.