# Never Look Back: Understanding Persistence in 3D Object Memory from Egocentric Videos

arXiv: https://arxiv.org/abs/2610.10538 · HTML: https://mlsanity.com/paper/2610.10538/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 62 references that are in the map.
- First posted on arXiv 7 Oct 2026.

## Builds on (most cited first, 20 of 20)

- [Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context](https://mlsanity.com/paper/2403.05530/) (2024)
- [SAM 3: Segment Anything with Concepts](https://mlsanity.com/paper/2511.16719/) (2025)
- [VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMs](https://mlsanity.com/paper/2406.07476/) (2024)
- [Depth Anything 3: Recovering the Visual Space from Any Views](https://mlsanity.com/paper/2511.10647/) (2025)
- [Cambrian-S: Towards Spatial Supersensing in Video](https://mlsanity.com/paper/2511.04670/) (2025)
- [FastVGGT: Training-Free Acceleration of Visual Geometry Transformer](https://mlsanity.com/paper/2509.02560/) (2025)
- [Online Video Understanding: OVBench and VideoChat-Online](https://mlsanity.com/paper/2501.00584/) (2024)
- [Khronos: A Unified Approach for Spatio-Temporal Metric-Semantic SLAM in Dynamic Environments](https://mlsanity.com/paper/2402.13817/) (2024)
- [WildDet3D: Scaling Promptable 3D Detection in the Wild](https://mlsanity.com/paper/2604.08626/) (2026)
- [EgoMemReason: A Memory-Driven Reasoning Benchmark for Long-Horizon Egocentric Video Understanding](https://mlsanity.com/paper/2605.09874/) (2026)
- [Online Episodic Memory Visual Query Localization with Egocentric Streaming Object Memory](https://mlsanity.com/paper/2411.16934/) (2024)
- [EgoAdapt: A Multi-Scene Egocentric Adaptation Method for CVPR 2026 HD-EPIC VQA Challenge](https://mlsanity.com/paper/2605.24500/) (2026)
- [From Pixels to Graphs: using Scene and Knowledge Graphs for HD-EPIC VQA Challenge](https://mlsanity.com/paper/2506.08553/) (2025)
- [GSMem: 3D Gaussian Splatting as Persistent Spatial Memory for Zero-Shot Embodied Exploration and Reasoning](https://mlsanity.com/paper/2603.19137/) (2026)
- [Keep It in Mind: User Centric Continual Spatial Intelligence Reasoning in Egocentric Video Streams](https://mlsanity.com/paper/2606.15200/) (2026)
- [MEMORA: Embodied Action Memory from Egocentric Videos for Reasoning and Planning](https://mlsanity.com/paper/2607.14252/) (2026)
- [EAGLE: Episodic Appearance- and Geometry-aware Memory for Unified 2D-3D Visual Query Localization in Egocentric Vision](https://mlsanity.com/paper/2511.08007/) (2025)
- [Whareformer: Learning to Track What is Where in Long Egocentric Videos](https://mlsanity.com/paper/2607.08537/) (2026)
- [Memory Over Maps: 3D Object Localization Without Reconstruction](https://mlsanity.com/paper/2603.20530/) (2026)
- [R4DSG: Relative 4D Scene Graph Memory for Object-Centric Question Answering in Long Egocentric Video](https://mlsanity.com/paper/2608.11017/) (2026)

## Abstract (arXiv)

As we move through the world and carry out everyday tasks, we encounter objects that may become relevant only later. We are capable of recalling where we left something or what was inside a container, even without knowing we would need it later. Here, we study how an embodied assistant can build a similar memory from egocentric videos, by observing a person's day-to-day activities. We present Ledger, a persistent 3D object memory that combines object locations, their histories, and contextual descriptions. It associates observations across the recording and retains objects after they leave the view, including those the person never touches. It clusters each object's observations by resting locations and records a move only after repeated evidence, reducing the effect of localization noise. Short descriptions preserve details such as an object's contents or supporting surface. It saves these records to later answer spatial questions without having to access the original images or video. Our memory raises HD-EPIC accuracy from 29.7% to 42.6%, UCS-Bench accuracy from 33.8% to 38.5% and localizes Ego4D objects with a 0.99 m median error on returned predictions. Our analyses identify complementary roles for temporal persistence, contextual descriptions, and retrieval. Our study on 100 stitched streams of multiple scenes each further exposes failures in both retrieval and construction. Per-scene construction partially recovers the performance lost across scene changes compared to that of single scene streams.