# A Controlled Audit of Personal AI Memory for Rating Prediction

arXiv: https://arxiv.org/abs/2610.02764 · HTML: https://mlsanity.com/paper/2610.02764/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 10 references that are in the map.
- First posted on arXiv 2 Oct 2026.

## Builds on (most cited first, 9 of 9)

- [Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory](https://mlsanity.com/paper/2504.19413/) (2025)
- [All Models are Wrong, but Many are Useful: Learning a Variable's Importance by Studying an Entire Class of Prediction Models Simultaneously](https://mlsanity.com/paper/1801.01489/) (2018)
- [Recommendations as Treatments: Debiasing Learning and Evaluation](https://mlsanity.com/paper/1602.05352/) (2016)
- [Language-Based User Profiles for Recommendation](https://mlsanity.com/paper/2402.15623/) (2024)
- [MemoryCD: Benchmarking Long-Context User Memory of LLM Agents for Lifelong Cross-Domain Personalization](https://mlsanity.com/paper/2603.25973/) (2026)
- [Can Large Language Models Understand Preferences in Personalized Recommendation?](https://mlsanity.com/paper/2501.13391/) (2025)
- [Memory Assisted LLM for Personalized Recommendation System](https://mlsanity.com/paper/2505.03824/) (2025)
- [MemRerank: Preference Memory for Personalized Product Reranking](https://mlsanity.com/paper/2603.29247/) (2026)
- [Does Your Agent's Memory Survive a Model Upgrade? A Controlled Study of Memory Portability](https://mlsanity.com/paper/2609.05339/) (2026)

## Abstract (arXiv)

In structured rating prediction, does a personal AI use historical item-rating associations, or mainly the user's rating tendencies? We audit this distinction by permuting historical ratings within each user while preserving the exact rating distribution, item support, and metadata. We combine this control with full history, native memory extraction, and matched numerical readers in a publicly frozen evaluation of 400 held-out user profiles and 6,160 target ratings across Coat and MovieLens. On Coat, the tested Qwen-written Mem0 pipeline increases user-macro mean absolute error relative to full history by 0.084 for Qwen and 0.149 for Phi; both family-adjusted bootstrap intervals exclude zero. Correct historical assignments help both readers on Coat, but the corresponding MovieLens effects are smaller and inconclusive after adjustment. A history-only ridge reader outperforms Qwen in both domains and Phi on MovieLens, while the Coat Phi comparison is unresolved. All 2,800 reader calls, including 150 invalid outputs, are retained under a fixed fallback rule. A separate implementation verifies inputs, metrics, and all ten primary contrasts. The contribution is a reproducible diagnostic study showing why extraction, association use, output reliability, and reader choice require separate evaluation.