# Know You Before You Speak: User-State Modeling for LLM Personalization in Multi-Turn Conversation

arXiv: https://arxiv.org/abs/2605.24647 · HTML: https://mlsanity.com/paper/2605.24647/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 57 references that are in the map.
- First posted on arXiv 23 May 2026.

## Builds on (most cited first, 30 of 39)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [MemoryBank: Enhancing Large Language Models with Long-Term Memory](https://mlsanity.com/paper/2305.10250/) (2023)
- [RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrieval](https://mlsanity.com/paper/2401.18059/) (2024)
- [A Survey on Dialogue Systems: Recent Advances and New Frontiers](https://mlsanity.com/paper/1711.01731/) (2017)
- [MemSkill: Learning and Evolving Memory Skills for Self-Evolving Agents](https://mlsanity.com/paper/2602.02474/) (2026)
- [Hello Again! LLM-powered Personalized Agent for Long-term Dialogue](https://mlsanity.com/paper/2406.05925/) (2024)
- [Aligning to Thousands of Preferences via System Message Generalization](https://mlsanity.com/paper/2405.17977/) (2024)
- [Large Language Models Empowered Personalized Web Agents](https://mlsanity.com/paper/2410.17236/) (2024)
- [Reinforced Latent Reasoning for LLM-based Recommendation](https://mlsanity.com/paper/2505.19092/) (2025)
- [Measuring What Makes You Unique: Difference-Aware User Modeling for Enhancing LLM Personalization](https://mlsanity.com/paper/2503.02450/) (2025)
- [Crafting Personalized Agents through Retrieval-Augmented Generation on Editable Memory Graphs](https://mlsanity.com/paper/2409.19401/) (2024)
- [HumanLM: Simulating Users with State Alignment Beats Response Imitation](https://mlsanity.com/paper/2603.03303/) (2026)
- [NextQuill: Causal Preference Modeling for Enhancing LLM Personalization](https://mlsanity.com/paper/2506.02368/) (2025)
- [Think-While-Generating: On-the-Fly Reasoning for Personalized Long-Form Generation](https://mlsanity.com/paper/2512.06690/) (2025)
- [Active Preference Inference using Language Models and Probabilistic Reasoning](https://mlsanity.com/paper/2312.12009/) (2023)
- [Enhancing Personalized Dialogue Generation with Contrastive Latent Variables: Combining Sparse and Dense Persona](https://mlsanity.com/paper/2305.11482/) (2023)
- [Latent Inter-User Difference Modeling for LLM Personalization](https://mlsanity.com/paper/2507.20849/) (2025)
- [A Personalized Dialogue Generator with Implicit User Persona Detection](https://mlsanity.com/paper/2204.07372/) (2022)
- [PRIME: Large Language Model Personalization with Cognitive Dual-Memory and Personalized Thought Process](https://mlsanity.com/paper/2507.04607/) (2025)
- [Explicit v.s. Implicit Memory: Exploring Multi-hop Complex Reasoning Over Personalized Information](https://mlsanity.com/paper/2508.13250/) (2025)
- [L-MTP: Leap Multi-Token Prediction Beyond Adjacent Context for Large Language Models](https://mlsanity.com/paper/2505.17505/) (2025)
- [Consistent Client Simulation for Motivational Interviewing-based Counseling](https://mlsanity.com/paper/2502.02802/) (2025)
- [RLPF: Reinforcement Learning from Prediction Feedback for User Summarization with LLMs](https://mlsanity.com/paper/2409.04421/) (2024)
- [NextMem: Towards Latent Factual Memory for LLM-based Agents](https://mlsanity.com/paper/2603.15634/) (2026)
- [MIRACLE: Towards Personalized Dialogue Generation with Latent-Space Multiple Personal Attribute Control](https://mlsanity.com/paper/2310.18342/) (2023)
- [Few-shot Dialogue Strategy Learning for Motivational Interviewing via Inductive Reasoning](https://mlsanity.com/paper/2403.15737/) (2024)
- [CAMI: A Counselor Agent Supporting Motivational Interviewing through State Inference and Topic Exploration](https://mlsanity.com/paper/2502.02807/) (2025)
- [Mem-PAL: Towards Memory-based Personalized Dialogue Assistants for Long-term User-Agent Interaction](https://mlsanity.com/paper/2511.13410/) (2025)
- [MSL: Not All Tokens Are What You Need for Tuning LLM as a Recommender](https://mlsanity.com/paper/2504.04178/) (2025)

## Built on by (most cited first, 3 of 3)

- [Mind2Dialogue: Training Human-Aware Language Models by Simulating User Mental States](https://mlsanity.com/paper/2609.15972/) (2026)
- [Relational Positioning as a Measurable Risk Object: History-Carried Lock-in and Self-Confabulation in Multi-Turn Human-AI Dialogue](https://mlsanity.com/paper/2607.11437/) (2026)
- [Rubric-Aware On-Policy Self-Distillation for LLM Personalization](https://mlsanity.com/paper/2609.35262/) (2026)

## Abstract (arXiv)

Personalized dialogue requires more than recalling explicit user histories: systems also need to infer hidden user states that evolve through interaction and shape appropriate response strategies. Existing memory- and profile-based methods primarily reuse observable user information, offering limited support for modeling user-state dynamics or selecting actions based on how they shape future user states. We propose PUMA (Prospective User-state Modeling for Action selection), a framework grounded in the Free Energy Principle (FEP) that formulates personalization as decision-making under partial observability, centered on an explicit user state model that captures latent user states and their action-conditioned dynamics. At each turn, PUMA maintains a belief over the user's hidden state, refines the user state model for observation generation and action-conditioned state transition, and selects dialogue actions by minimizing expected free energy, balancing epistemic and pragmatic objectives under a unified criterion. This formulation shifts personalization from passive memory retrieval to model-based decision-making over user evolution. We instantiate PUMA on healthcare-oriented counseling and motivational interviewing benchmarks with latent state annotations for rigorous evaluation. Experiments show that PUMA improves long-horizon dialogue outcomes while maintaining strong response quality, and a cross-dataset study demonstrates more reliable user-state estimation and next-state prediction.