mlsanityData to 14 Oct 2026

SimGuide: Typed Multi-Context User Representations for Preference-Conditioned Agent Planning

arXiv 2608.24888 ↗First posted 14 May 2026cs.AIcited by 0

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

Agents that act on a user's behalf must plan differently for different users, and increasingly do so from some structured representation of user context and not from raw interaction history. How much that structure is worth, and which parts of it carry the value, is largely unmeasured. We introduce SimBench, 47 preference-conditioned planning tasks over 9 synthetic users represented as 28 typed, potentially conflicting context blocks, where the correct plan depends on which contexts are active and how their conflicts are resolved. Against it we evaluate SimGuide, a framework combining typed multi-context representation, explicit conflict arbitration, and optional procedural grounding of individual constraints. Across three models and six user-context representations, SimGuide's typed blocks with arbitration outperform retrieval over the same user's past decisions by +0.210, +0.205 and +0.144 Preference Adherence on Llama 3.3 70B, GPT-4o and Claude Sonnet 4.5 respectively (all p < 0.001); removing the arbitration instruction alone costs up to +0.209. Grounding each constraint with a worked example of past application helps only where the model has headroom: +0.094 on Llama 70B (p < 0.001), falling to +0.029 on GPT-4o and +0.002 on Claude, which already scores perfectly on 35 of 47 tasks without it. We report the benchmark's minimum detectable effect alongside its results. The benchmark ships with a provenance audit that re-derives every reported number from the prompt that produced it.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 15 of its 29 references that are in the map. First posted on arXiv 14 May 2026.

Builds on

15 of 29 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains17 Jun

Other references · 14

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  4. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  5. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  6. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,568
  7. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 2,006
  8. 2023MemGPT: Towards LLMs as Operating Systemscited by 1,389
  9. 2023LaMP: When Large Language Models Meet Personalizationcited by 437
  10. 2020With Little Power Comes Great Responsibilitycited by 142
  11. 2024Personalized Pieces: Efficient Personalized Large Language Models through Collaborative Effortscited by 74
  12. 2024PersonalLLM: Tailoring LLMs to Individual Preferencescited by 60
2 more
  1. 2024LLM-Personalize: Aligning LLM Planners with Human Preferences via Reinforced Self-Training for Housekeeping Robotscited by 18
  2. 2025Instant Personalized Large Language Model Adaptation via Hypernetworkcited by 18

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025UserBench: An Interactive Gym Environment for User-Centric Agents
  2. 2026AcCoRD: Evaluating User-Agent Collaboration Under Realistic User Preference Dynamics
  3. 2026UserProxyBench: Evaluating LLM User Simulators for Agent Benchmarks and Training
  4. 2026PrefIx: Understand and Adapt to User Preference in Human-Agent Interaction
  5. 2025Inferring the Unspoken: Aligning Embodied Agents with Implicit Preferences
  6. 2026PERSONAJUDGE: Simulating Individual Human Preference Judgments with Evaluator-Specific Demonstration Data
  7. 2026Optimizing Sparse Outcomes Through Dense Behavioral Signals via Value-Guided Preference Distillation
  8. 2026UserLM-R1: Modeling Human Reasoning in User Language Models with Multi-Reward Reinforcement Learning