HUSH-Bench: Measuring Memory-Use Boundaries for Sensitive History in Conversational Agents
Not on the map. Why?
Abstract (arXiv)
Long-term memory helps conversational agents maintain continuity across sessions, while relevance and current-turn warrant remain distinct decisions. We study this boundary under a stated conservative policy in which sensitive history shapes a response when the current turn supplies a reason to use it. We introduce HUSH-Bench, a controlled benchmark of 2,400 benign prompts paired with histories containing one marked sensitive disclosure and matched no-memory references. HUSH-Bench measures unsolicited history integration with the Unsolicited History Integration Score (UIS; 0--100, higher is worse), records whether the marked disclosure reaches the generator, and includes paired prompts that differ only in whether the user asks the assistant to use earlier context. We evaluate four models under no-memory, full-context, and three retrieval-based memory settings. Memory access raises UIS from near zero to 8.9--26.6 for one model and 51.3--83.0 for the other three. Retrieval systems expose the marked disclosure in 23.0%--30.3% of cases, while related sensitive entries or summaries remain available and three models continue to show high UIS. Across four generators, an explicit invitation increases target-memory uptake scores by 27.0--41.3; measured helpfulness remains stable while mean over-scope rises. These results motivate treating memory storage, retrieval, warrant, and per-turn scope as separate design decisions.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 21 of its 38 references that are in the map. First posted on arXiv 4 Jun 2026.
Builds on
21 of 38 references.
References · 21
- 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,429
- 2025A-MEM: Agentic Memory for LLM Agentscited by 1,063
- 2026DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligencecited by 1,040
- 2020Beyond Accuracy: Behavioral Testing of NLP models with CheckListcited by 1,006
- 2024Evaluating Very Long-Term Conversational Memory of LLM Agentscited by 830
- 2024Personal LLM Agents: Insights and Survey about the Capability, Efficiency and Securitycited by 260
- 2025Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactionscited by 254
- 2024SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusalcited by 240
- 2025Know Me, Respond to Me: Benchmarking LLMs for Dynamic User Profiling and Personalized Responses at Scalecited by 178
- 2025Do LLMs Recognize Your Preferences? Evaluating Personalized Preference Following in LLMscited by 148
- 2025MemOS: A Memory OS for AI Systemcited by 138
- 2025In Prospect and Retrospect: Reflective Memory Management for Long-term Personalized Dialogue Agentscited by 106
9 more
- 2025Unveiling Privacy Risks in LLM Agent Memorycited by 101
- 2025How Memory Management Impacts LLM Agents: An Empirical Study of Experience-Following Behaviorcited by 101
- 2024ALERT: A Comprehensive Benchmark for Assessing Large Language Models' Safety through Red Teamingcited by 98
- 2024Large Language Models Empowered Personalized Web Agentscited by 66
- 2024Crafting Personalized Agents through Retrieval-Augmented Generation on Editable Memory Graphscited by 38
- 2025Teaching Language Models to Evolve with Users: Dynamic Profile Modeling for Personalized Alignmentcited by 24
- 2026OP-Bench: Benchmarking Over-Personalization for Memory-Augmented Personalized Conversational Agentscited by 22
- 2025Personalized Safety in LLMs: A Benchmark and A Planning-Based Agent Approachcited by 16
- 2026When Personalization Legitimizes Risks: Uncovering Safety Vulnerabilities in Personalized Dialogue Agentscited by 7
Built on by
Citing papers per year: 2026: 2.
2 papers.