mlsanityData to 7 Oct 2026

Is First Person Vision Challenging for Object Tracking?

arXiv 2108.13665 ↗First posted 31 Aug 2021cs.CVcited by 21

Not on the map. Why?

Abstract (arXiv)

Understanding human-object interactions is fundamental in First Person Vision (FPV). Tracking algorithms which follow the objects manipulated by the camera wearer can provide useful cues to effectively model such interactions. Visual tracking solutions available in the computer vision literature have significantly improved their performance in the last years for a large variety of target objects and tracking scenarios. However, despite a few previous attempts to exploit trackers in FPV applications, a methodical analysis of the performance of state-of-the-art trackers in this domain is still missing. In this paper, we fill the gap by presenting the first systematic study of object tracking in FPV. Our study extensively analyses the performance of recent visual trackers and baseline FPV trackers with respect to different aspects and considering a new performance measure. This is achieved through TREK-150, a novel benchmark dataset composed of 150 densely annotated video sequences. Our results show that object tracking in FPV is challenging, which suggests that more research efforts should be devoted to this problem so that tracking could benefit FPV tasks.

Key facts. Cited by 21 papers in the map (data through 7 Oct 2026). Builds on 62 of its 110 references that are in the map. First posted on arXiv 31 Aug 2021.

Builds on

62 of 110 references.

References · 62

  1. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  2. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,829
  3. 2017Simple Online and Realtime Tracking with a Deep Association Metriccited by 908
  4. 2018Scaling Egocentric Vision: The EPIC-KITCHENS Datasetcited by 789
  5. 2016Fully-Convolutional Siamese Networks for Object Trackingcited by 566
  6. 2018GOT-10k: A Large High-Diversity Benchmark for Generic Object Tracking in the Wildcited by 471
  7. 2018SiamRPN++: Evolution of Siamese Visual Tracking with Very Deep Networkscited by 445
  8. 2018LaSOT: A High-quality Benchmark for Large-scale Single Object Trackingcited by 424
  9. 2019Learning Discriminative Model Prediction for Trackingcited by 344
  10. 2018TrackingNet: A Large-Scale Dataset and Benchmark for Object Tracking in the Wildcited by 342
  11. 2018ATOM: Accurate Tracking by Overlap Maximizationcited by 285
  12. 2020Understanding Human Hands in Contact at Internet Scalecited by 281
38 more
  1. 2018Fast Online Object Tracking and Segmentation: A Unifying Approachcited by 272
  2. 2016ECO: Efficient Convolution Operators for Trackingcited by 239
  3. 2018Long-Term Feature Banks for Detailed Video Understandingcited by 239
  4. 2020Probabilistic Regression for Visual Trackingcited by 231
  5. 2020Ocean: Object-aware Anchor-free Trackingcited by 212
  6. 2015Learning Multi-Domain Convolutional Neural Networks for Visual Trackingcited by 209
  7. 2020Siamese Box Adaptive Network for Visual Trackingcited by 189
  8. 2019SiamFC++: Towards Robust and Accurate Visual Tracking with Target Estimation Guidelinescited by 177
  9. 2019Deeper and Wider Siamese Networks for Real-Time Visual Trackingcited by 156
  10. 2020Reconstructing Hand-Object Interactions in the Wildcited by 143
  11. 2017Need for Speed: A Benchmark for Higher Frame Rate Object Trackingcited by 137
  12. 2020MOTChallenge: A Benchmark for Single-Camera Multiple Target Trackingcited by 134
  13. 2020Know Your Surroundings: Exploiting Scene Information for Object Trackingcited by 119
  14. 2020Rolling-Unrolling LSTMs for Action Anticipation from First-Person Videocited by 119
  15. 2015A Novel Performance Evaluation Methodology for Single-Target Trackerscited by 113
  16. 2019What Would You Expect? Anticipating Egocentric Actions with Rolling-Unrolling LSTMs and Modality Attentioncited by 104
  17. 2016Learning to Track at 100 FPS with Deep Regression Networkscited by 89
  18. 2020Learning What to Learn for Video Object Segmentationcited by 89
  19. 2017Real-time Hand Tracking under Occlusion from an Egocentric RGB-D Sensorcited by 84
  20. 2020High-Performance Long-Term Tracking with Meta-Updatercited by 79
  21. 2020The MECCANO Dataset: Understanding Human-Object Interactions from Egocentric Videos in an Industrial-like Domaincited by 79
  22. 2019Learning the Model Update for Siamese Trackerscited by 75
  23. 2019D3S -- A Discriminative Single Shot Segmentation Trackercited by 69
  24. 2017Learning Background-Aware Correlation Filters for Visual Trackingcited by 65
  25. 2018VITAL: VIsual Tracking via Adversarial Learningcited by 65
  26. 2018Learning Spatial-Temporal Regularized Correlation Filters for Visual Trackingcited by 63
  27. 2015Staple: Complementary Learners for Real-Time Trackingcited by 60
  28. 2019GlobalTrack: A Simple and Strong Baseline for Long-term Trackingcited by 57
  29. 2018Long-term Tracking in the Wild: A Benchmarkcited by 55
  30. 2019Next-Active-Object prediction from Egocentric Videoscited by 55
  31. 2016Going Deeper into First-Person Activity Recognitioncited by 48
  32. 2016Discriminative Scale Space Trackingcited by 46
  33. 2021Predicting the Future from First Person (Egocentric) Vision: A Surveycited by 36
  34. 2018Meta-Tracker: Fast and Robust Online Adaptation for Visual Object Trackerscited by 34
  35. 2019'Skimming-Perusal' Tracking: A Framework for Real-Time and Robust Long-term Trackingcited by 34
  36. 2019CDTB: A Color and Depth Visual Object Tracking Dataset and Benchmarkcited by 31
  37. 2020Symbiotic Attention with Privileged Information for Egocentric Action Recognitioncited by 27
  38. 2015You-Do, I-Learn: Unsupervised Multi-User egocentric Approach Towards Video-Based Guidancecited by 17

50 most cited of 62 All 62 (JSON)

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 4; 2023: 10; 2024: 3; 2025: 3.

21 papers.

Sort:

  1. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,530
  2. 2025EgoLife: Towards Egocentric Life Assistantcited by 86
  3. 2022MECCANO: A Multimodal Egocentric Dataset for Humans Behavior Understanding in the Industrial-like Domaincited by 36
  4. 2022Egocentric Prediction of Action Target in 3Dcited by 31
  5. 2023EgoTracks: A Long-term Egocentric Visual Object Tracking Datasetcited by 27
  6. 2023Instance Tracking in 3D Scenes from Egocentric Videoscited by 17
  7. 2024Online Episodic Memory Visual Query Localization with Egocentric Streaming Object Memorycited by 11
  8. 2023360VOT: A New Benchmark Dataset for Omnidirectional Visual Object Trackingcited by 10
  9. 2023Localizing Active Objects from Egocentric Vision with Symbolic World Knowledgecited by 8
  10. 2024360VOTS: Visual Object Tracking and Segmentation in Omnidirectional Videoscited by 8
  11. 2024Ego3DT: Tracking Every 3D Object in Ego-centric Videoscited by 7
  12. 2023Active Open-Vocabulary Recognition: Let Intelligent Moving Mitigate CLIP Limitationscited by 6
9 more
  1. 2023Tracking Skiers from the Top to the Bottomcited by 6
  2. 2022PVT++: A Simple End-to-End Latency-Aware Visual Tracking Frameworkcited by 5
  3. 2022CoCoLoT: Combining Complementary Trackers in Long-Term Visual Trackingcited by 2
  4. 2025HERO-VQL: Hierarchical, Egocentric and Robust Visual Query Localizationcited by 2
  5. 2023Visualizing Skiers' Trajectories in Monocular Videoscited by 1
  6. 2023ZJU ReLER Submission for EPIC-KITCHEN Challenge 2023: TREK-150 Single Object Trackingcited by 1
  7. 2025Is Tracking really more challenging in First Person Egocentric Vision?cited by 1
  8. 2023Bayesian Decision Making to Localize Visual Queries in 2Dcited by 0
  9. 2023EXOT: Exit-aware Object Tracker for Safe Robotic Manipulation of Moving Objectcited by 0