# Sensible Agent: A Framework for Unobtrusive Interaction with Proactive AR Agents

arXiv: https://arxiv.org/abs/2509.09255 · HTML: https://mlsanity.com/paper/2509.09255/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Sep 2025.

## Built on by (most cited first, 15 of 15)

- [Dive into Claude Code: The Design Space of Today's and Future AI Agent Systems](https://mlsanity.com/paper/2604.14228/) (2026)
- [ProAgent: Harnessing On-Demand Sensory Contexts for Proactive LLM Agent Systems in the Wild](https://mlsanity.com/paper/2512.06721/) (2025)
- [VisionClaw: Always-On AI Agents through Smart Glasses](https://mlsanity.com/paper/2604.03486/) (2026)
- [Vinci2: Providing Proactive Assistance in Continuous Egocentric Videos](https://mlsanity.com/paper/2607.11523/) (2026)
- [SpeechLess: Micro-utterance with Personalized Spatial Memory-aware Assistant in Everyday Augmented Reality](https://mlsanity.com/paper/2602.00793/) (2026)
- [Memento: Towards Proactive Visualization of Everyday Memories with Personal Wearable AR Assistant](https://mlsanity.com/paper/2601.17622/) (2026)
- [From Speech-to-Spatial: Grounding Utterances on A Live Shared View with Augmented Reality](https://mlsanity.com/paper/2602.03059/) (2026)
- [EgoIntent: A Pre-Outcome Micro-Step Benchmark for Understanding What, Why, and Next](https://mlsanity.com/paper/2603.12147/) (2026)
- [Agents in the Large: Perception-Centered Architecture for Persistent Agents](https://mlsanity.com/paper/2608.30478/) (2026)
- [From Passive Feeds to Guided Discovery: AI-Initiated Interaction for Vague Intent in Content Exploration](https://mlsanity.com/paper/2605.02902/) (2026)
- [SenseWalk: Agent-Based Semantic Trajectory Simulation Powered by Large Language Models in Zoned Environments](https://mlsanity.com/paper/2607.00989/) (2026)
- [Unified Agent: Managing Interactions across Devices](https://mlsanity.com/paper/2608.05729/) (2026)
- [DocuTeam: Mixed-Initiative Multi-Agent Discussions around Evolving Documents](https://mlsanity.com/paper/2609.29309/) (2026)
- [A Benchmarking Framework for Context-aware XR Interfaces](https://mlsanity.com/paper/2609.30466/) (2026)
- [VoCa: Designing Speech-Canvas Interaction for Voice-Based Conversational Agents](https://mlsanity.com/paper/2610.04706/) (2026)

## Abstract (arXiv)

Proactive AR agents promise context-aware assistance, but their interactions often rely on explicit voice prompts or responses, which can be disruptive or socially awkward. We introduce Sensible Agent, a framework designed for unobtrusive interaction with these proactive agents. Sensible Agent dynamically adapts both "what" assistance to offer and, crucially, "how" to deliver it, based on real-time multimodal context sensing. Informed by an expert workshop (n=12) and a data annotation study (n=40), the framework leverages egocentric cameras, multimodal sensing, and Large Multimodal Models (LMMs) to infer context and suggest appropriate actions delivered via minimally intrusive interaction modes. We demonstrate our prototype on an XR headset through a user study (n=10) in both AR and VR scenarios. Results indicate that Sensible Agent significantly reduces perceived interaction effort compared to voice-prompted baseline, while maintaining high usability and achieving higher preference.