VisionClaw: Always-On AI Agents through Smart Glasses
On the map: LLM agents and tool use → Multimodal and embodied agents
Abstract (arXiv)
We present VisionClaw, an always-on wearable AI agent that integrates live egocentric perception with agentic task execution. Running on Meta Ray-Ban smart glasses, VisionClaw continuously perceives real-world context and enables in-situ, speech-driven action initiation and delegation via OpenClaw AI agents. Therefore, users can directly execute tasks through the smart glasses, such as adding real-world objects to an Amazon cart, generating notes from physical documents, receiving meeting briefings on the go, creating events from posters, or controlling IoT devices. We evaluate VisionClaw through a controlled laboratory study (N=12) and a longitudinal deployment study (N=5). Results show that integrating perception and execution enables faster task completion and reduces interaction overhead compared to non-always-on and non-agent baselines. Beyond performance gains, deployment findings reveal a shift in interaction: tasks are initiated opportunistically during ongoing activities, and execution is increasingly delegated rather than manually controlled. These results suggest a new paradigm for wearable AI agents, where perception and action are continuously coupled to support situated, hands-free interaction.
Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 41 of its 82 references that are in the map. First posted on arXiv 3 Apr 2026.
Builds on
41 of 82 references.
Closest ancestors · 1
Same direction, newest first
Other references · 40
- 2020Scaling Laws for Neural Language Modelscited by 6,411
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
- 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
- 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 1,908
- 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,530
- 2023CogAgent: A Visual Language Model for GUI Agentscited by 692
- 2023Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectivescited by 477
- 2023AppAgent: Multimodal Agents as Smartphone Userscited by 407
- 2024Mobile-Agent: Autonomous Multi-Modal Mobile Device Agent with Visual Perceptioncited by 283
- 2024OS-Copilot: Towards Generalist Computer Agents with Self-Improvementcited by 203
- 2023Project Aria: A New Tool for Egocentric Multi-Modal AI Researchcited by 201
- 2023HoloAssist: an Egocentric Human Interaction Dataset for Interactive AI Assistants in the Real Worldcited by 159
28 more
- 2025EgoLife: Towards Egocentric Life Assistantcited by 86
- 2024GazePointAR: A Context-Aware Multimodal Voice Assistant for Pronoun Disambiguation in Wearable Augmented Realitycited by 35
- 2024AMEGO: Active Memory from long EGOcentric videoscited by 35
- 2024WorldScribe: Towards Context-Aware Live Visual Descriptionscited by 26
- 2024Satori: Towards Proactive AR Assistant with Belief-Desire-Intention User Modelingcited by 24
- 2024Memoro: Using Large Language Models to Realize a Concise Interface for Real-Time Memory Augmentationcited by 20
- 2025Morae: Proactively Pausing UI Agents for User Choicescited by 18
- 2023XAIR: A Framework of Explainable AI in Augmented Realitycited by 17
- 2024GazeGPT: Augmenting Human Capabilities using Gaze-contingent Contextual AI for Smart Eyewearcited by 15
- 2025Sensible Agent: A Framework for Unobtrusive Interaction with Proactive AR Agentscited by 14
- 2025ProMemAssist: Exploring Timely Proactive Assistance Through Working Memory Modeling in Multi-Modal Wearable Devicescited by 13
- 2024Augmented Object Intelligence with XR-Objectscited by 12
- 2025Vid2Coach: Transforming How-To Videos into Task Assistantscited by 12
- 2025AiGet: Transforming Everyday Moments into Hidden Knowledge Discovery with AI Assistance on Smart Glassescited by 11
- 2024OmniActions: Predicting Digital Actions in Response to Real-World Multimodal Sensory Inputs with LLMscited by 10
- 2025GazeLLM: Multimodal LLMs incorporating Human Visual Attentioncited by 9
- 2026PEARL: Personalized Streaming Video Understanding Modelcited by 8
- 2025Guided Reality: Generating Visually-Enriched AR Task Guidance with LLMs and Vision Modelscited by 7
- 2025Mirai: A Wearable Proactive AI "Inner-Voice" for Contextual Nudgingcited by 6
- 2024Augmented Conversation with Embedded Speech-Driven On-the-Fly Referencing in ARcited by 4
- 2025EgoPrivacy: What Your First-Person Camera Says About You?cited by 4
- 2025From Following to Understanding: Investigating the Role of Reflective Prompts in AR-Guided Tasks to Promote Task Understandingcited by 3
- 2026SpeechLess: Micro-utterance with Personalized Spatial Memory-aware Assistant in Everyday Augmented Realitycited by 3
- 2026Gazeify Then Voiceify: Physical Object Referencing Through Gaze and Voice Interaction with Displayless Smart Glassescited by 2
- 2026Memento: Towards Proactive Visualization of Everyday Memories with Personal Wearable AR Assistantcited by 1
- 2026From Speech-to-Spatial: Grounding Utterances on A Live Shared View with Augmented Realitycited by 1
- 2026Proactive Conversational Assistant for a Procedural Manual Task based on Audio and IMUcited by 1
- 2026Beyond the Desk: Barriers and Future Opportunities for AI to Assist Scientists in Embodied Physical Taskscited by 1
Built on by
Citing papers per year: 2026: 4.
4 papers.
- 2026SpeakerLLM: A Speaker-Specialized Audio-LLM for Speaker Understanding and Verification Reasoningcited by 2
- 2026VisualClaw: A Real-Time, Personalized Agent for the Physical Worldcited by 2
- 2026Position: Life-Logging Video Streams Make the Privacy-Utility Trade-off Inevitablecited by 0
- 2026AI Smart Glasses for Wearable Intelligence: From Egocentric Sensing to Agentic Personalizationcited by 0
- 2026AI Smart Glasses for Wearable Intelligence: From Egocentric Sensing to Agentic Personalization17 Sep 2026
- 2026VisualClaw: A Real-Time, Personalized Agent for the Physical World15 Jun 2026
- 2026SpeakerLLM: A Speaker-Specialized Audio-LLM for Speaker Understanding and Verification Reasoning14 May 2026
- 2026Position: Life-Logging Video Streams Make the Privacy-Utility Trade-off Inevitable11 May 2026
Similar, not cited
Close in wording, no citation either way
- 2026ABot-Claw: A Foundation for Persistent, Cooperative, and Self-Evolving Robotic Agents
- 2026Egocentric Co-Pilot: Web-Native Smart-Glasses Agents for Assistive Egocentric AI
- 2026ETA: A New Agentic Paradigm for Embodied Tasks
- 2025PhysiAgent: An Embodied Agent Framework in Physical World
- 2026LIBERO-Agent: Evaluating General-Purpose Agents for Direct Embodied Manipulation
- 2026EgoBench: An Interactive Egocentric Multimodal Benchmark for Tool-Using Agents
- 2026Exploratory, Communicative, and Deployable: Vision-Driven Embodied Agents for Open-World Mobile Manipulation
- 2026KnowAct-GUIClaw: Know Deeply, Act Perfectly, Personal GUI Assistant with Self-Evolving Memory and Skill