mlsanityData to 14 Oct 2026

Fewer Clarifications, Better Code: Benchmarking Cross-Session Personalized Ambiguity Adaptation in Coding Assistants

arXiv 2607.26611 ↗First posted 29 Jul 2026cs.AI cs.HCcited by 0

Not on the map. Why?

Abstract (arXiv)

AI-assisted coding increasingly translates informal user intent into executable software, yet coding requests often contain ambiguities that recur in user-specific ways across tasks and sessions. Existing disambiguation methods typically address each ambiguous request in isolation within the current coding session, often through eliciting additional clarification. However, whether resolved session history from the same user can serve as memory for resolving recurring personalized ambiguity in a newly opened session remains underexplored. We formulate personalized ambiguity adaptation as a new task: given a user's previously resolved coding sessions and a new ambiguous request, an assistant should identify the recurring ambiguity pattern, produce the intended executable solution, and minimize clarification. To benchmark this task, we introduce CAPA, which characterizes personalized coding ambiguity through six mechanisms and injects these mechanisms into unambiguous executable tasks using a controlled three-stage generation pipeline. CAPA contains 600 coding sessions across 60 balanced user--ambiguity cells, including 300 held-out evaluation sessions. We evaluate 12 recent LLMs under no-history and same-user-history conditions using executable success, first-turn success, and turns-to-completion. Our analyses examine task difficulty, user identity, and memory-based history use, and we further propose same-user history gating as a lightweight inference-time method. CAPA provides a foundation for developing long-term coding assistants that better align generated code with user intent while reducing repeated clarification.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 36 of its 49 references that are in the map. First posted on arXiv 29 Jul 2026.

Builds on

36 of 49 references.

References · 36

  1. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  2. 2021Program Synthesis with Large Language Modelscited by 3,637
  3. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
  4. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,391
  5. 2025A-MEM: Agentic Memory for LLM Agentscited by 1,063
  6. 2024Evaluating Very Long-Term Conversational Memory of LLM Agentscited by 830
  7. 2024LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memorycited by 707
  8. 2024WildChat: 1M ChatGPT Interaction Logs in the Wildcited by 629
  9. 2023LaMP: When Large Language Models Meet Personalizationcited by 437
  10. 2020AmbigQA: Answering Ambiguous Open-domain Questionscited by 390
  11. 2025Memory in the Age of AI Agentscited by 303
  12. 2022ASQA: Factoid Questions Meet Long-Form Answerscited by 234
24 more
  1. 2025Know Me, Respond to Me: Benchmarking LLMs for Dynamic User Profiling and Personalized Responses at Scalecited by 178
  2. 2020Open-Domain Question Answering Goes Conversational via Question Rewritingcited by 148
  3. 2023We're Afraid Language Models Aren't Modeling Ambiguitycited by 126
  4. 2023Clarify When Necessary: Resolving Ambiguity Through Interaction with LMscited by 98
  5. 2024Modeling Future Conversation Turns to Teach LLMs to Ask Clarifying Questionscited by 76
  6. 2024Aligning Language Models to Explicitly Handle Ambiguitycited by 56
  7. 2026From Recall to Forgetting: Benchmarking Long-Term Memory for Personalized Agentscited by 36
  8. 2023Asking Clarification Questions to Handle Ambiguity in Open-Domain QAcited by 26
  9. 2024HumanEvalComm: Benchmarking the Communication Competence of Code Generation for LLMs and LLM Agentcited by 24
  10. 2020Resolving Intent Ambiguities by Retrieving Discriminative Clarifying Questionscited by 16
  11. 2019Identifying Unclear Questions in Community Question Answering Websitescited by 12
  12. 2024Diversify-verify-adapt: Efficient and Robust Retrieval-Augmented Ambiguous Question Answeringcited by 10
  13. 2025Detecting Ambiguities to Guide Query Rewrite for Robust Conversations in Enterprise AI Assistantscited by 8
  14. 2024A Taxonomy of Ambiguity Types for NLPcited by 7
  15. 2025Disambiguation in Conversational Question Answering in the Era of LLMs and Agents: A Surveycited by 6
  16. 2026ClarifyCodeBench: Evaluating LLMs on Clarifying Ambiguous Requirements for Code Generationcited by 5
  17. 2026Clarity Is Not Assumed: Understanding LLM-Based Code Generation under Ambiguous Requirementscited by 3
  18. 2026Personalize-then-Store: Benchmarking and Learning Personalized Memory for Long-horizon Agentscited by 3
  19. 2026Beyond Retrieval: Learning Compact User Representations for Scalable LLM Personalizationcited by 3
  20. 2026Latent Preference Modeling for Multi-Session Personalized Tool Callingcited by 2
  21. 2026Momento: Evaluating Persistent Memory and Reasoning with Multi-Session Agentic Conversationscited by 2
  22. 2026PersonaTree: Structured Lifecycle Memory for Person Understanding in LLM Agentscited by 2
  23. 2026Learning User-Aware Recall: Personalized Retrieval in Long-Term Conversational Memorycited by 2
  24. 2026Latent Personal Memory: Represent personal memory as dynamic soft promptscited by 1

Built on by

Not cited yet.