# Memento: Towards Proactive Visualization of Everyday Memories with Personal Wearable AR Assistant

arXiv: https://arxiv.org/abs/2601.17622 · HTML: https://mlsanity.com/paper/2601.17622/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 80 references that are in the map.
- First posted on arXiv 24 Jan 2026.

## Builds on (most cited first, 30 of 40)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023)
- [From Local to Global: A Graph RAG Approach to Query-Focused Summarization](https://mlsanity.com/paper/2404.16130/) (2024)
- [Kosmos-2: Grounding Multimodal Large Language Models to the World](https://mlsanity.com/paper/2306.14824/) (2023)
- [Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphs](https://mlsanity.com/paper/1603.09320/) (2016)
- [YOLO-World: Real-Time Open-Vocabulary Object Detection](https://mlsanity.com/paper/2401.17270/) (2024)
- [MuRAG: Multimodal Retrieval-Augmented Generator for Open Question Answering over Images and Text](https://mlsanity.com/paper/2210.02928/) (2022)
- [EgoLife: Towards Egocentric Life Assistant](https://mlsanity.com/paper/2503.03803/) (2025)
- [User-LLM: Efficient LLM Contextualization with User Embeddings](https://mlsanity.com/paper/2402.13598/) (2024)
- [Embodied-RAG: General Non-parametric Embodied Memory for Retrieval and Generation](https://mlsanity.com/paper/2409.18313/) (2024)
- [LLMR: Real-time Prompting of Interactive Worlds using Large Language Models](https://mlsanity.com/paper/2309.12276/) (2023)
- [GazePointAR: A Context-Aware Multimodal Voice Assistant for Pronoun Disambiguation in Wearable Augmented Reality](https://mlsanity.com/paper/2404.08213/) (2024)
- [Proactive Conversational Agents with Inner Thoughts](https://mlsanity.com/paper/2501.00383/) (2024)
- [Text-to-Viz: Automatic Generation of Infographics from Proportion-Related Natural Language Statements](https://mlsanity.com/paper/1907.09091/) (2019)
- [WorldScribe: Towards Context-Aware Live Visual Descriptions](https://mlsanity.com/paper/2408.06627/) (2024)
- [Satori: Towards Proactive AR Assistant with Belief-Desire-Intention User Modeling](https://mlsanity.com/paper/2410.16668/) (2024)
- [CAESURA: Language Models as Multi-Modal Query Planners](https://mlsanity.com/paper/2308.03424/) (2023)
- [Memoro: Using Large Language Models to Realize a Concise Interface for Real-Time Memory Augmentation](https://mlsanity.com/paper/2403.02135/) (2024)
- [Vinci: A Real-time Embodied Smart Assistant based on Egocentric Vision-Language Model](https://mlsanity.com/paper/2412.21080/) (2024)
- [From Data to Story: Towards Automatic Animated Data Video Creation with LLM-based Multi-Agent Systems](https://mlsanity.com/paper/2408.03876/) (2024)
- [Sensible Agent: A Framework for Unobtrusive Interaction with Proactive AR Agents](https://mlsanity.com/paper/2509.09255/) (2025)
- [SUQL: Conversational Search over Structured and Unstructured Data with Large Language Models](https://mlsanity.com/paper/2311.09818/) (2023)
- [Augmented Object Intelligence with XR-Objects](https://mlsanity.com/paper/2404.13274/) (2024)
- [AiGet: Transforming Everyday Moments into Hidden Knowledge Discovery with AI Assistance on Smart Glasses](https://mlsanity.com/paper/2501.16240/) (2025)
- [OmniActions: Predicting Digital Actions in Response to Real-World Multimodal Sensory Inputs with LLMs](https://mlsanity.com/paper/2405.03901/) (2024)
- [OmniQuery: Contextually Augmenting Captured Multimodal Memory to Enable Personal Question Answering](https://mlsanity.com/paper/2409.08250/) (2024)
- [Encode-Store-Retrieve: Augmenting Human Memory through Language-Encoded Egocentric Perception](https://mlsanity.com/paper/2308.05822/) (2023)
- [Design Patterns for Situated Visualization in Augmented Reality](https://mlsanity.com/paper/2307.09157/) (2023)
- [Explainable XR: Understanding User Behaviors of XR Environments using LLM-assisted Analytics Framework](https://mlsanity.com/paper/2501.13778/) (2025)

## Built on by (most cited first, 1 of 1)

- [VisionClaw: Always-On AI Agents through Smart Glasses](https://mlsanity.com/paper/2604.03486/) (2026)

## Abstract (arXiv)

We introduce Memento, a conversational AR assistant that permanently captures and memorizes user's verbal queries alongside their spatiotemporal and activity contexts. By storing these "memories," Memento discovers connections between users' recurring interests and the contexts that trigger them. Upon detection of similar or identical spatiotemporal activity, Memento proactively recalls user interests and delivers up-to-date responses through AR, seamlessly integrating AR experience into their daily routine. Unlike prior work, each interaction in Memento is not a transient event, but a connected series of interactions with coherent long--term perspective, tailored to the user's broader multimodal (visual, spatial, temporal, and embodied) context. We conduct a preliminary evaluation through user feedbacks with participants of diverse expertise in immersive apps, and explore the value of proactive context-aware AR assistant in everyday settings. We share our findings and challenges in designing a proactive, context-aware AR system.