mlsanityData to 7 Oct 2026

SpeechLess: Micro-utterance with Personalized Spatial Memory-aware Assistant in Everyday Augmented Reality

arXiv 2602.00793 ↗First posted 31 Jan 2026cs.HC cs.CL cs.ETcited by 3

Not on the map. Why?

Abstract (arXiv)

Speaking aloud to a wearable AR assistant in public can be socially awkward, and re-articulating the same requests every day creates unnecessary effort. We present SpeechLess, a wearable AR assistant that introduces a speech-based intent granularity control paradigm grounded in personalized spatial memory. SpeechLess helps users "speak less," while still obtaining the information they need, and supports gradual explicitation of intent when more complex expression is required. SpeechLess binds prior interactions to multimodal personal context-space, time, activity, and referents-to form spatial memories, and leverages them to extrapolate missing intent dimensions from under-specified user queries. This enables users to dynamically adjust how explicitly they express their informational needs, from full-utterance to micro/zero-utterance interaction. We motivate our design through a week-long formative study using a commercial smart glasses platform, revealing discomfort with public voice use, frustration with repetitive speech, and hardware constraints. Building on these insights, we design SpeechLess, and evaluate it through controlled lab and in-the-wild studies. Our results indicate that regulated speech-based interaction, can improve everyday information access, reduce articulation effort, and support socially acceptable use without substantially degrading perceived usability or intent resolution accuracy across diverse everyday environments.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 37 of its 77 references that are in the map. First posted on arXiv 31 Jan 2026.

Builds on

37 of 77 references.

References · 37

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  2. 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spacescited by 580
  3. 2023Project Aria: A New Tool for Egocentric Multi-Modal AI Researchcited by 201
  4. 2025Seeing, Listening, Remembering, and Reasoning: A Multimodal Agent with Long-Term Memorycited by 88
  5. 2024User-LLM: Efficient LLM Contextualization with User Embeddingscited by 74
  6. 2024Ask-before-Plan: Proactive Language Agents for Real-World Planningcited by 59
  7. 2025Embodied AI Agents: Modeling the Worldcited by 57
  8. 2025ContextAgent: Context-Aware Proactive LLM Agents with Open-World Sensory Perceptionscited by 44
  9. 2024Aria Everyday Activities Datasetcited by 38
  10. 2024GazePointAR: A Context-Aware Multimodal Voice Assistant for Pronoun Disambiguation in Wearable Augmented Realitycited by 35
  11. 2024Proactive Conversational Agents with Inner Thoughtscited by 35
  12. 2024WorldScribe: Towards Context-Aware Live Visual Descriptionscited by 26
25 more
  1. 2024Satori: Towards Proactive AR Assistant with Belief-Desire-Intention User Modelingcited by 24
  2. 2024Memoro: Using Large Language Models to Realize a Concise Interface for Real-Time Memory Augmentationcited by 20
  3. 2013Context Aware Computing for The Internet of Things: A Surveycited by 18
  4. 2023XAIR: A Framework of Explainable AI in Augmented Realitycited by 17
  5. 2025Sensible Agent: A Framework for Unobtrusive Interaction with Proactive AR Agentscited by 14
  6. 2025ProMemAssist: Exploring Timely Proactive Assistance Through Working Memory Modeling in Multi-Modal Wearable Devicescited by 13
  7. 2024Augmented Object Intelligence with XR-Objectscited by 12
  8. 2025AiGet: Transforming Everyday Moments into Hidden Knowledge Discovery with AI Assistance on Smart Glassescited by 11
  9. 2024OmniActions: Predicting Digital Actions in Response to Real-World Multimodal Sensory Inputs with LLMscited by 10
  10. 2024OmniQuery: Contextually Augmenting Captured Multimodal Memory to Enable Personal Question Answeringcited by 10
  11. 2025Intention-Guided Cognitive Reasoning for Egocentric Long-Term Action Anticipationcited by 10
  12. 2023Encode-Store-Retrieve: Augmenting Human Memory through Language-Encoded Egocentric Perceptioncited by 9
  13. 2023Design Patterns for Situated Visualization in Augmented Realitycited by 6
  14. 2025Explainable XR: Understanding User Behaviors of XR Environments using LLM-assisted Analytics Frameworkcited by 6
  15. 2025Mirai: A Wearable Proactive AI "Inner-Voice" for Contextual Nudgingcited by 6
  16. 2025EgoTrigger: Toward Audio-Driven Image Capture for Human Memory Enhancement in All-Day Energy-Efficient Smart Glassescited by 6
  17. 2025Vision-Based Multimodal Interfaces: A Survey and Taxonomy for Enhanced Context-Aware System Designcited by 5
  18. 2025Wearable AR in Everyday Contexts: Insights from a Digital Ethnography of YouTube Videoscited by 4
  19. 2023Context-Dependent Memory in Situated Visualizationcited by 3
  20. 2024SituationalLLM: Proactive language models with scene awareness for dynamic, contextual task guidancecited by 3
  21. 2025PersoNo: Personalised Notification Urgency Classifier in Mixed Realitycited by 3
  22. 2024Towards Context-Aware Adaptation in Extended Reality: A Design Space for XR Interfaces and an Adaptive Placement Strategycited by 2
  23. 2025Augmented Reality Productivity In-the-Wild: A Diary Study of Usage Patterns and Experiences of Working With AR Laptops in Real-World Settingscited by 1
  24. 2025Rethinking Privacy Indicators in Extended Reality: Multimodal Design for Situationally Impaired Bystanderscited by 1
  25. 2025PILAR: Personalizing Augmented Reality Interactions with LLM-based Human-Centric and Trustworthy Explanations for Daily Use Casescited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.