Cognitive Prosthetic: An AI-Enabled Multimodal System for Episodic Recall in Knowledge Work
Not on the map. Why?
Abstract (arXiv)
Modern knowledge workplaces increasingly strain human episodic memory as individuals navigate fragmented attention, overlapping meetings, and multimodal information streams. Existing workplace tools provide partial support through note-taking or analytics but rarely integrate cognitive, physiological, and attentional context into retrievable memory representations. This paper presents the Cognitive Prosthetic Multimodal System (CPMS) --an AI-enabled proof-of-concept designed to support episodic recall in knowledge work through structured episodic capture and natural language retrieval. CPMS synchronizes speech transcripts, physiological signals, and gaze behavior into temporally aligned, JSON-based episodic records processed locally for privacy. Beyond data logging, the system includes a web-based retrieval interface that allows users to query past workplace experiences using natural language, referencing semantic content, time, attentional focus, or physiological state. We present CPMS as a functional proof-of-concept demonstrating the technical feasibility of transforming heterogeneous sensor data into queryable episodic memories. The system is designed to be modular, supporting operation with partial sensor configurations, and incorporates privacy safeguards for workplace deployment. This work contributes an end-to-end, privacy-aware architecture for AI-enabled memory augmentation in workplace settings.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 11 of its 53 references that are in the map. First posted on arXiv 2 Mar 2026.
Builds on
11 of 53 references.
References · 11
- 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,530
- 2021VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Textcited by 427
- 2022Multimodal Learning with Transformers: A Surveycited by 399
- 2024Learning from Observer Gaze:Zero-Shot Attention Prediction Oriented by Human-Object Interaction Recognitioncited by 17
- 2024AlzheimerRAG: Multimodal Retrieval Augmented Generation for Clinical Use Cases using PubMed articlescited by 3
- 2020Ten Questions in Lifelog Mining and Information Recallcited by 2
- 2019DFS: A Dataset File System for Data Discovering Userscited by 1
- 2020Audiovisual Speech-In-Noise (SIN) Performance of Young Adults with ADHDcited by 1
- 2022StreamingHub: Interactive Stream Analysis Workflowscited by 1
- 2025Beyond Gaze Overlap: Analyzing Joint Visual Attention Dynamics Using Egocentric Datacited by 1
- 2026Towards Affordable, Non-Invasive Real-Time Hypoglycemia Detection Using Wearable Sensor Signalscited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.