# SpeechLess: Micro-utterance with Personalized Spatial Memory-aware Assistant in Everyday Augmented Reality

arXiv: https://arxiv.org/abs/2602.00793 · HTML: https://mlsanity.com/paper/2602.00793/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 77 references that are in the map.
- First posted on arXiv 31 Jan 2026.

## Builds on (most cited first, 30 of 37)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spaces](https://mlsanity.com/paper/2412.14171/) (2024)
- [Project Aria: A New Tool for Egocentric Multi-Modal AI Research](https://mlsanity.com/paper/2308.13561/) (2023)
- [Seeing, Listening, Remembering, and Reasoning: A Multimodal Agent with Long-Term Memory](https://mlsanity.com/paper/2508.09736/) (2025)
- [User-LLM: Efficient LLM Contextualization with User Embeddings](https://mlsanity.com/paper/2402.13598/) (2024)
- [Embodied AI Agents: Modeling the World](https://mlsanity.com/paper/2506.22355/) (2025)
- [Ask-before-Plan: Proactive Language Agents for Real-World Planning](https://mlsanity.com/paper/2406.12639/) (2024)
- [ContextAgent: Context-Aware Proactive LLM Agents with Open-World Sensory Perceptions](https://mlsanity.com/paper/2505.14668/) (2025)
- [Aria Everyday Activities Dataset](https://mlsanity.com/paper/2402.13349/) (2024)
- [GazePointAR: A Context-Aware Multimodal Voice Assistant for Pronoun Disambiguation in Wearable Augmented Reality](https://mlsanity.com/paper/2404.08213/) (2024)
- [Proactive Conversational Agents with Inner Thoughts](https://mlsanity.com/paper/2501.00383/) (2024)
- [WorldScribe: Towards Context-Aware Live Visual Descriptions](https://mlsanity.com/paper/2408.06627/) (2024)
- [Satori: Towards Proactive AR Assistant with Belief-Desire-Intention User Modeling](https://mlsanity.com/paper/2410.16668/) (2024)
- [Memoro: Using Large Language Models to Realize a Concise Interface for Real-Time Memory Augmentation](https://mlsanity.com/paper/2403.02135/) (2024)
- [Context Aware Computing for The Internet of Things: A Survey](https://mlsanity.com/paper/1305.0982/) (2013)
- [XAIR: A Framework of Explainable AI in Augmented Reality](https://mlsanity.com/paper/2303.16292/) (2023)
- [Sensible Agent: A Framework for Unobtrusive Interaction with Proactive AR Agents](https://mlsanity.com/paper/2509.09255/) (2025)
- [ProMemAssist: Exploring Timely Proactive Assistance Through Working Memory Modeling in Multi-Modal Wearable Devices](https://mlsanity.com/paper/2507.21378/) (2025)
- [Augmented Object Intelligence with XR-Objects](https://mlsanity.com/paper/2404.13274/) (2024)
- [AiGet: Transforming Everyday Moments into Hidden Knowledge Discovery with AI Assistance on Smart Glasses](https://mlsanity.com/paper/2501.16240/) (2025)
- [OmniActions: Predicting Digital Actions in Response to Real-World Multimodal Sensory Inputs with LLMs](https://mlsanity.com/paper/2405.03901/) (2024)
- [OmniQuery: Contextually Augmenting Captured Multimodal Memory to Enable Personal Question Answering](https://mlsanity.com/paper/2409.08250/) (2024)
- [Intention-Guided Cognitive Reasoning for Egocentric Long-Term Action Anticipation](https://mlsanity.com/paper/2508.01742/) (2025)
- [Encode-Store-Retrieve: Augmenting Human Memory through Language-Encoded Egocentric Perception](https://mlsanity.com/paper/2308.05822/) (2023)
- [Design Patterns for Situated Visualization in Augmented Reality](https://mlsanity.com/paper/2307.09157/) (2023)
- [Explainable XR: Understanding User Behaviors of XR Environments using LLM-assisted Analytics Framework](https://mlsanity.com/paper/2501.13778/) (2025)
- [Mirai: A Wearable Proactive AI "Inner-Voice" for Contextual Nudging](https://mlsanity.com/paper/2502.02370/) (2025)
- [EgoTrigger: Toward Audio-Driven Image Capture for Human Memory Enhancement in All-Day Energy-Efficient Smart Glasses](https://mlsanity.com/paper/2508.01915/) (2025)
- [Vision-Based Multimodal Interfaces: A Survey and Taxonomy for Enhanced Context-Aware System Design](https://mlsanity.com/paper/2501.13443/) (2025)
- [Wearable AR in Everyday Contexts: Insights from a Digital Ethnography of YouTube Videos](https://mlsanity.com/paper/2502.06191/) (2025)

## Built on by (most cited first, 3 of 3)

- [VisionClaw: Always-On AI Agents through Smart Glasses](https://mlsanity.com/paper/2604.03486/) (2026)
- [EgoCITE: Context-Augmented Indexing and Time-Aware Retrieval for Long-Horizon Egocentric Memory](https://mlsanity.com/paper/2608.12627/) (2026)
- [AI Smart Glasses for Wearable Intelligence: From Egocentric Sensing to Agentic Personalization](https://mlsanity.com/paper/2609.19793/) (2026)

## Abstract (arXiv)

Speaking aloud to a wearable AR assistant in public can be socially awkward, and re-articulating the same requests every day creates unnecessary effort. We present SpeechLess, a wearable AR assistant that introduces a speech-based intent granularity control paradigm grounded in personalized spatial memory. SpeechLess helps users "speak less," while still obtaining the information they need, and supports gradual explicitation of intent when more complex expression is required. SpeechLess binds prior interactions to multimodal personal context-space, time, activity, and referents-to form spatial memories, and leverages them to extrapolate missing intent dimensions from under-specified user queries. This enables users to dynamically adjust how explicitly they express their informational needs, from full-utterance to micro/zero-utterance interaction. We motivate our design through a week-long formative study using a commercial smart glasses platform, revealing discomfort with public voice use, frustration with repetitive speech, and hardware constraints. Building on these insights, we design SpeechLess, and evaluate it through controlled lab and in-the-wild studies. Our results indicate that regulated speech-based interaction, can improve everyday information access, reduce articulation effort, and support socially acceptable use without substantially degrading perceived usability or intent resolution accuracy across diverse everyday environments.