# Predicting Important Objects for Egocentric Video Summarization

arXiv: https://arxiv.org/abs/1505.04803 · HTML: https://mlsanity.com/paper/1505.04803/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 May 2015.

## Built on by (most cited first, 21 of 21)

- [Ego4D: Around the World in 3,000 Hours of Egocentric Video](https://mlsanity.com/paper/2110.07058/) (2021)
- [Socratic Models: Composing Zero-Shot Multimodal Reasoning with Language](https://mlsanity.com/paper/2204.00598/) (2022)
- [Affordance Diffusion: Synthesizing Hand-Object Interactions](https://mlsanity.com/paper/2303.12538/) (2023)
- [Ego-Exo: Transferring Visual Representations from Third-person to First-person Videos](https://mlsanity.com/paper/2104.07905/) (2021)
- [E2(GO)MOTION: Motion Augmented Event Stream for Egocentric Action Recognition](https://mlsanity.com/paper/2112.03596/) (2021)
- [NaQ: Leveraging Narrations as Queries to Supervise Episodic Memory](https://mlsanity.com/paper/2301.00746/) (2023)
- [Beyond the Contact: Discovering Comprehensive Affordance for 3D Objects from Pre-trained 2D Diffusion Models](https://mlsanity.com/paper/2401.12978/) (2024)
- [Egocentric Prediction of Action Target in 3D](https://mlsanity.com/paper/2203.13116/) (2022)
- [EgoTracks: A Long-term Egocentric Visual Object Tracking Dataset](https://mlsanity.com/paper/2301.03213/) (2023)
- [Learning from Semantic Alignment between Unpaired Multiviews for Egocentric Video Recognition](https://mlsanity.com/paper/2308.11489/) (2023)
- [Hierarchical Multimodal Transformer to Summarize Videos](https://mlsanity.com/paper/2109.10559/) (2021)
- [Building a Mind Palace: Structuring Environment-Grounded Semantic Graphs for Effective Long Video Analysis with LLMs](https://mlsanity.com/paper/2501.04336/) (2025)
- [SpotEM: Efficient Video Search for Episodic Memory](https://mlsanity.com/paper/2306.15850/) (2023)
- [GPT4Ego: Unleashing the Potential of Pre-trained Models for Zero-Shot Egocentric Action Recognition](https://mlsanity.com/paper/2401.10039/) (2024)
- [Which Viewpoint Shows it Best? Language for Weakly Supervising View Selection in Multi-view Instructional Videos](https://mlsanity.com/paper/2411.08753/) (2024)
- [AudioVisual Video Summarization](https://mlsanity.com/paper/2105.07667/) (2021)
- [DAViD: Modeling Dynamic Affordance of 3D Objects Using Pre-trained Video Diffusion Models](https://mlsanity.com/paper/2501.08333/) (2025)
- [Your Interest, Your Summaries: Query-Focused Long Video Summarization](https://mlsanity.com/paper/2410.14087/) (2024)
- [Contextual Guided Segmentation Framework for Semi-supervised Video Instance Segmentation](https://mlsanity.com/paper/2106.03330/) (2021)
- [PhilHumans: Benchmarking Machine Learning for Personal Health](https://mlsanity.com/paper/2405.02770/) (2024)
- [A Framework for Egocentric and Exocentric Procedural Understanding via Temporal Segmentation and Semantic Abstraction](https://mlsanity.com/paper/2610.00069/) (2026)

## Abstract (arXiv)

We present a video summarization approach for egocentric or "wearable" camera data. Given hours of video, the proposed method produces a compact storyboard summary of the camera wearer's day. In contrast to traditional keyframe selection techniques, the resulting summary focuses on the most important objects and people with which the camera wearer interacts. To accomplish this, we develop region cues indicative of high-level saliency in egocentric video---such as the nearness to hands, gaze, and frequency of occurrence---and learn a regressor to predict the relative importance of any new region based on these cues. Using these predictions and a simple form of temporal event detection, our method selects frames for the storyboard that reflect the key object-driven happenings. We adjust the compactness of the final summary given either an importance selection criterion or a length budget; for the latter, we design an efficient dynamic programming solution that accounts for importance, visual uniqueness, and temporal displacement. Critically, the approach is neither camera-wearer-specific nor object-specific; that means the learned importance metric need not be trained for a given user or context, and it can predict the importance of objects and people that have never been seen previously. Our results on two egocentric video datasets show the method's promise relative to existing techniques for saliency and summarization.