# Trespassing the Boundaries: Labeling Temporal Bounds for Object Interactions in Egocentric Video

arXiv: https://arxiv.org/abs/1703.09026 · HTML: https://mlsanity.com/paper/1703.09026/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Mar 2017.

## Built on by (most cited first, 14 of 14)

- [Ego4D: Around the World in 3,000 Hours of Egocentric Video](https://mlsanity.com/paper/2110.07058/) (2021)
- [On Semantic Similarity in Video Retrieval](https://mlsanity.com/paper/2103.10095/) (2021)
- [BasicTAD: an Astounding RGB-Only Baseline for Temporal Action Detection](https://mlsanity.com/paper/2205.02717/) (2022)
- [Generative Adversarial Network for Future Hand Segmentation from Egocentric Video](https://mlsanity.com/paper/2203.11305/) (2022)
- [FineBio: A Fine-Grained Video Dataset of Biological Experiments with Hierarchical Annotation](https://mlsanity.com/paper/2402.00293/) (2024)
- [Learning Action Changes by Measuring Verb-Adverb Textual Relationships](https://mlsanity.com/paper/2303.15086/) (2023)
- [Are you Struggling? Dataset and Baselines for Struggle Determination in Assembly Videos](https://mlsanity.com/paper/2402.11057/) (2024)
- [A Generalized & Robust Framework For Timestamp Supervision in Temporal Action Segmentation](https://mlsanity.com/paper/2207.10137/) (2022)
- [An Action Is Worth Multiple Words: Handling Ambiguity in Action Recognition](https://mlsanity.com/paper/2210.04933/) (2022)
- [Inductive Attention for Video Action Anticipation](https://mlsanity.com/paper/2212.08830/) (2022)
- [Video Editing for Video Retrieval](https://mlsanity.com/paper/2402.02335/) (2024)
- [DRIFT: A Residual Flow Adapter for Decoding Continuous Outputs in Vision-Language Models](https://mlsanity.com/paper/2606.05758/) (2026)
- [Egocentric Activity Recognition and Localization on a 3D Map](https://mlsanity.com/paper/2105.09544/) (2021)
- [When Does a Laugh Begin? Structured Annotator Disagreement in Temporal Laughter Localization](https://mlsanity.com/paper/2609.06646/) (2026)

## Abstract (arXiv)

Manual annotations of temporal bounds for object interactions (i.e. start and end times) are typical training input to recognition, localization and detection algorithms. For three publicly available egocentric datasets, we uncover inconsistencies in ground truth temporal bounds within and across annotators and datasets. We systematically assess the robustness of state-of-the-art approaches to changes in labeled temporal bounds, for object interaction recognition. As boundaries are trespassed, a drop of up to 10% is observed for both Improved Dense Trajectories and Two-Stream Convolutional Neural Network. We demonstrate that such disagreement stems from a limited understanding of the distinct phases of an action, and propose annotating based on the Rubicon Boundaries, inspired by a similarly named cognitive model, for consistent temporal bounds of object interactions. Evaluated on a public dataset, we report a 4% increase in overall accuracy, and an increase in accuracy for 55% of classes when Rubicon Boundaries are used for temporal annotations.