mlsanityData to 7 Oct 2026

Personalizing Large Language Model Agents with Small Policy Models

arXiv 2608.00215 ↗First posted 31 Jul 2026cs.AIcited by 0

On the map: LLM agents and tool use → Agent memory systems

Abstract (arXiv)

Large language model (LLM) agents can retrieve memory, call tools, ask clarifying questions, and vary response style, yet adapting these execution decisions to an individual user remains difficult. Fine-tuning a separate LLM is costly or impossible for proprietary systems, while prompts and memory primarily expose user information to the agent rather than adapt its execution decisions from feedback. We formulate personalization of a frozen agent as online learning of a per-user execution policy from scalar feedback observed only for the executed action. We propose FABLE (Factorized Adaptive Bandit Layer for Execution), a lightweight policy layer outside a potentially black-box host agent. FABLE factorizes memory, information-acquisition, and response decisions so feedback updates related choices; filters actions through an externally specified feasible set before exploration; and learns user-specific residual preferences relative to a fixed default-and-cost score via Bayesian contextual Thompson sampling. Under a linear residual-reward model, a calibrated variant inherits an expected-regret bound against the best feasible action. We also characterize preferences unidentifiable under persistent feasibility constraints and provide anytime-valid false-promotion control. Across personalized-reasoning, controlled-feedback, and executable tool-use evaluations, FABLE improves several preference-sensitive behaviors relative to rule-only control while remaining competitive on end-to-end task performance.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Agent memory systems, one of 16 directions in LLM agents and tool use. Builds on 28 of its 43 references that are in the map. First posted on arXiv 31 Jul 2026.

Builds on

28 of 43 references.

Closest ancestors · 6

Same direction, newest first

  1. 2026Memory as a Controlled Process: Learned Adaptive Memory Management for LLM Agents15 Jul
  2. 2026MemToolAgent: Leveraging Memory for Tool Using Agents Based on Environment and User Feedback6 Jun
  3. 2026Latent Preference Modeling for Multi-Session Personalized Tool Calling20 Apr
  4. 2026User Preference Modeling for Conversational LLM Agents: Weak Rewards from Retrieval-Augmented Interaction21 Mar
  5. 2025PersonaAgent: Bridging Memory and Action for Personalized LLM Agents6 Jun
  6. 2025A-MEM: Agentic Memory for LLM Agents17 Feb

Other references · 22

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  2. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  3. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  4. 2023Let's Verify Step by Stepcited by 3,598
  5. 2023MemGPT: Towards LLMs as Operating Systemscited by 1,295
  6. 2023LaMP: When Large Language Models Meet Personalizationcited by 425
  7. 2024Personalizing Reinforcement Learning from Human Feedback with Variational Preference Learningcited by 147
  8. 2023Clarify When Necessary: Resolving Ambiguity Through Interaction with LMscited by 93
  9. 2024RLHF from Heterogeneous Feedback via Personalization and Preference Aggregationcited by 71
  10. 2016Conservative Contextual Linear Banditscited by 56
  11. 2019Safe Linear Thompson Sampling with Side Informationcited by 37
  12. 2022Contextual Bandits with Large Action Spaces: Made Practicalcited by 32
10 more
  1. 2025Amulet: ReAlignment During Test Time for Personalized Preference Adaptation of LLMscited by 29
  2. 2020IntelligentPooling: Practical Thompson Sampling for mHealthcited by 24
  3. 2017Action Centered Contextual Banditscited by 23
  4. 2019Warm-starting Contextual Bandits: Robustly Combining Supervised and Bandit Feedbackcited by 21
  5. 2026Learning Personalized Agents from Human Feedbackcited by 16
  6. 2018Factored Banditscited by 7
  7. 2025T-POP: Test-Time Personalization with Online Preference Feedbackcited by 6
  8. 2025ADAPT: Actively Discovering and Adapting to Preferences for any Taskcited by 5
  9. 2026SAGER: Self-Evolving User Policy Skills for Recommendation Agentcited by 2
  10. 2026OLIVIA: Online Learning via Inference-time Action Adaptation for Decision Making in LLM ReAct Agentscited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026MAPLE: A Sub-Agent Architecture for Memory, Learning, and Personalization in Agentic AI Systems
  2. 2026CAPTURE: Disentangling Preference Drift from Memory Poisoning in Personalized LLM Agents
  3. 2023Retroformer: Retrospective Large Language Agents with Policy Gradient Optimization
  4. 2024Personalized Adaptation via In-Context Preference Learning
  5. 2025Memory-R1: Enhancing Large Language Model Agents to Manage and Utilize Memories via Reinforcement Learning
  6. 2025Language Model Personalization via Reward Factorization
  7. 2025Memento: Fine-tuning LLM Agents without Fine-tuning LLMs
  8. 2026AdaMem: Learning What to Remember with Adaptive Memory Policies for Personalized Agents