mlsanityData to 7 Oct 2026

VisionClaw: Always-On AI Agents through Smart Glasses

arXiv 2604.03486 ↗First posted 3 Apr 2026cs.HC cs.AI cs.CVcited by 4

On the map: LLM agents and tool use → Multimodal and embodied agents

Abstract (arXiv)

We present VisionClaw, an always-on wearable AI agent that integrates live egocentric perception with agentic task execution. Running on Meta Ray-Ban smart glasses, VisionClaw continuously perceives real-world context and enables in-situ, speech-driven action initiation and delegation via OpenClaw AI agents. Therefore, users can directly execute tasks through the smart glasses, such as adding real-world objects to an Amazon cart, generating notes from physical documents, receiving meeting briefings on the go, creating events from posters, or controlling IoT devices. We evaluate VisionClaw through a controlled laboratory study (N=12) and a longitudinal deployment study (N=5). Results show that integrating perception and execution enables faster task completion and reduces interaction overhead compared to non-always-on and non-agent baselines. Beyond performance gains, deployment findings reveal a shift in interaction: tasks are initiated opportunistically during ongoing activities, and execution is increasingly delegated rather than manually controlled. These results suggest a new paradigm for wearable AI agents, where perception and action are continuously coupled to support situated, hands-free interaction.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 41 of its 82 references that are in the map. First posted on arXiv 3 Apr 2026.

Builds on

41 of 82 references.

Closest ancestors · 1

Same direction, newest first

  1. 2026Ego2Web: A Web Agent Benchmark Grounded in Egocentric Videos23 Mar

Other references · 40

  1. 2020Scaling Laws for Neural Language Modelscited by 6,411
  2. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  3. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  4. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 1,908
  5. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,530
  6. 2023CogAgent: A Visual Language Model for GUI Agentscited by 692
  7. 2023Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectivescited by 477
  8. 2023AppAgent: Multimodal Agents as Smartphone Userscited by 407
  9. 2024Mobile-Agent: Autonomous Multi-Modal Mobile Device Agent with Visual Perceptioncited by 283
  10. 2024OS-Copilot: Towards Generalist Computer Agents with Self-Improvementcited by 203
  11. 2023Project Aria: A New Tool for Egocentric Multi-Modal AI Researchcited by 201
  12. 2023HoloAssist: an Egocentric Human Interaction Dataset for Interactive AI Assistants in the Real Worldcited by 159
28 more
  1. 2025EgoLife: Towards Egocentric Life Assistantcited by 86
  2. 2024GazePointAR: A Context-Aware Multimodal Voice Assistant for Pronoun Disambiguation in Wearable Augmented Realitycited by 35
  3. 2024AMEGO: Active Memory from long EGOcentric videoscited by 35
  4. 2024WorldScribe: Towards Context-Aware Live Visual Descriptionscited by 26
  5. 2024Satori: Towards Proactive AR Assistant with Belief-Desire-Intention User Modelingcited by 24
  6. 2024Memoro: Using Large Language Models to Realize a Concise Interface for Real-Time Memory Augmentationcited by 20
  7. 2025Morae: Proactively Pausing UI Agents for User Choicescited by 18
  8. 2023XAIR: A Framework of Explainable AI in Augmented Realitycited by 17
  9. 2024GazeGPT: Augmenting Human Capabilities using Gaze-contingent Contextual AI for Smart Eyewearcited by 15
  10. 2025Sensible Agent: A Framework for Unobtrusive Interaction with Proactive AR Agentscited by 14
  11. 2025ProMemAssist: Exploring Timely Proactive Assistance Through Working Memory Modeling in Multi-Modal Wearable Devicescited by 13
  12. 2024Augmented Object Intelligence with XR-Objectscited by 12
  13. 2025Vid2Coach: Transforming How-To Videos into Task Assistantscited by 12
  14. 2025AiGet: Transforming Everyday Moments into Hidden Knowledge Discovery with AI Assistance on Smart Glassescited by 11
  15. 2024OmniActions: Predicting Digital Actions in Response to Real-World Multimodal Sensory Inputs with LLMscited by 10
  16. 2025GazeLLM: Multimodal LLMs incorporating Human Visual Attentioncited by 9
  17. 2026PEARL: Personalized Streaming Video Understanding Modelcited by 8
  18. 2025Guided Reality: Generating Visually-Enriched AR Task Guidance with LLMs and Vision Modelscited by 7
  19. 2025Mirai: A Wearable Proactive AI "Inner-Voice" for Contextual Nudgingcited by 6
  20. 2024Augmented Conversation with Embedded Speech-Driven On-the-Fly Referencing in ARcited by 4
  21. 2025EgoPrivacy: What Your First-Person Camera Says About You?cited by 4
  22. 2025From Following to Understanding: Investigating the Role of Reflective Prompts in AR-Guided Tasks to Promote Task Understandingcited by 3
  23. 2026SpeechLess: Micro-utterance with Personalized Spatial Memory-aware Assistant in Everyday Augmented Realitycited by 3
  24. 2026Gazeify Then Voiceify: Physical Object Referencing Through Gaze and Voice Interaction with Displayless Smart Glassescited by 2
  25. 2026Memento: Towards Proactive Visualization of Everyday Memories with Personal Wearable AR Assistantcited by 1
  26. 2026From Speech-to-Spatial: Grounding Utterances on A Live Shared View with Augmented Realitycited by 1
  27. 2026Proactive Conversational Assistant for a Procedural Manual Task based on Audio and IMUcited by 1
  28. 2026Beyond the Desk: Barriers and Future Opportunities for AI to Assist Scientists in Embodied Physical Taskscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 4.

4 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026ABot-Claw: A Foundation for Persistent, Cooperative, and Self-Evolving Robotic Agents
  2. 2026Egocentric Co-Pilot: Web-Native Smart-Glasses Agents for Assistive Egocentric AI
  3. 2026ETA: A New Agentic Paradigm for Embodied Tasks
  4. 2025PhysiAgent: An Embodied Agent Framework in Physical World
  5. 2026LIBERO-Agent: Evaluating General-Purpose Agents for Direct Embodied Manipulation
  6. 2026EgoBench: An Interactive Egocentric Multimodal Benchmark for Tool-Using Agents
  7. 2026Exploratory, Communicative, and Deployable: Vision-Driven Embodied Agents for Open-World Mobile Manipulation
  8. 2026KnowAct-GUIClaw: Know Deeply, Act Perfectly, Personal GUI Assistant with Self-Evolving Memory and Skill