mlsanityData to 7 Oct 2026

EPIC-KITCHENS VISOR Benchmark: VIdeo Segmentations and Object Relations

arXiv 2209.13064 ↗First posted 26 Sep 2022cs.CV cs.AI cs.LGcited by 139

Not on the map. Why?

Abstract (arXiv)

We introduce VISOR, a new dataset of pixel annotations and a benchmark suite for segmenting hands and active objects in egocentric video. VISOR annotates videos from EPIC-KITCHENS, which comes with a new set of challenges not encountered in current video segmentation datasets. Specifically, we need to ensure both short- and long-term consistency of pixel-level annotations as objects undergo transformative interactions, e.g. an onion is peeled, diced and cooked - where we aim to obtain accurate pixel-level annotations of the peel, onion pieces, chopping board, knife, pan, as well as the acting hands. VISOR introduces an annotation pipeline, AI-powered in parts, for scalability and quality. In total, we publicly release 272K manual semantic masks of 257 object classes, 9.9M interpolated dense masks, 67K hand-object relations, covering 36 hours of 179 untrimmed videos. Along with the annotations, we introduce three challenges in video object segmentation, interaction understanding and long-term reasoning. For data, code and leaderboards: http://epic-kitchens.github.io/VISOR

Key facts. Cited by 139 papers in the map (data through 7 Oct 2026). Builds on 33 of its 81 references that are in the map. First posted on arXiv 26 Sep 2022.

Builds on

33 of 81 references.

References · 33

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  2. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
  3. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
  4. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,530
  5. 2019LVIS: A Dataset for Large Vocabulary Instance Segmentationcited by 1,215
  6. 2017The 2017 DAVIS Challenge on Video Object Segmentationcited by 942
  7. 2018Scaling Egocentric Vision: The EPIC-KITCHENS Datasetcited by 789
  8. 2016Hollywood in Homes: Crowdsourcing Data Collection for Activity Understandingcited by 535
  9. 2017AVA: A Video Dataset of Spatio-temporally Localized Atomic Visual Actionscited by 449
  10. 2019PointRend: Image Segmentation as Renderingcited by 439
  11. 2020GRAB: A Dataset of Whole-Body Human Grasping of Objectscited by 397
  12. 2018YouTube-VOS: A Large-Scale Video Object Segmentation Benchmarkcited by 381
21 more
  1. 2019Learning joint reconstruction of hands and manipulated objectscited by 377
  2. 2020Understanding Human Hands in Contact at Internet Scalecited by 281
  3. 2019Action Genome: Actions as Composition of Spatio-temporal Scene Graphscited by 249
  4. 2017Detecting and Recognizing Human-Object Interactionscited by 199
  5. 2021GOAL: Generating 4D Whole-Body Motion for Hand-Object Graspingcited by 142
  6. 2021Unidentified Video Objects: A Benchmark for Dense, Open-World Segmentationcited by 133
  7. 2018Efficient Interactive Annotation of Segmentation Datasets with Polygon-RNN++cited by 119
  8. 2022Joint Hand Motion and Interaction Hotspots Prediction from Egocentric Videoscited by 110
  9. 2019Large-scale interactive object segmentation with human annotatorscited by 109
  10. 2019Fast Interactive Object Annotation with Curve-GCNcited by 95
  11. 2017Annotating Object Instances with a Polygon-RNNcited by 88
  12. 2020The AVA-Kinetics Localized Human Actions Video Datasetcited by 82
  13. 2020The MECCANO Dataset: Understanding Human-Object Interactions from Egocentric Videos in an Industrial-like Domaincited by 79
  14. 2018Object Level Visual Reasoning in Videoscited by 67
  15. 2021Human Hands as Probes for Interactive Object Understandingcited by 51
  16. 2022Grasp'D: Differentiable Contact-rich Grasp Synthesis for Multi-fingered Handscited by 49
  17. 2022Grounding Answers for Visual Questions Asked by Visually Impaired Peoplecited by 47
  18. 2021D3D-HOI: Dynamic 3D Human-Object Interactions from Videoscited by 39
  19. 2020Detecting Hands and Recognizing Physical Contact in the Wildcited by 38
  20. 2022MECCANO: A Multimodal Egocentric Dataset for Humans Behavior Understanding in the Industrial-like Domaincited by 36
  21. 2019Contextual Attention for Hand Detection in the Wildcited by 16

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 4; 2023: 35; 2024: 28; 2025: 41; 2026: 31.

139 papers.

Sort:

By area:

  1. 2023Segment Anythingcited by 8,158
  2. 2024SAM 2: Segment Anything in Images and Videoscited by 2,784
  3. 2023Reconstructing Hands in 3D with Transformerscited by 298
  4. 2023Affordances from Human Videos as a Versatile Representation for Roboticscited by 221
  5. 2025HD-EPIC: A Highly-Detailed Egocentric Video Datasetcited by 101
  6. 2024EgoExoLearn: A Dataset for Bridging Asynchronous Ego- and Exo-centric View of Procedural Activities in Real Worldcited by 93
  7. 2025EgoLife: Towards Egocentric Life Assistantcited by 86
  8. 2023An Outlook into the Future of Egocentric Visioncited by 84
  9. 2025VideoPainter: Any-length Video Inpainting and Editing with Plug-and-Play Context Controlcited by 83
  10. 2022LVOS: A Benchmark for Long-term Video Object Segmentationcited by 76
  11. 2023Towards Generalizable Zero-Shot Manipulation via Translating Human Interaction Planscited by 51
  12. 2023Action Scene Graphs for Long-Form Understanding of Egocentric Videoscited by 51
38 more
  1. 2023EPIC Fields: Marrying 3D Geometry and Video Understandingcited by 49
  2. 2022Breaking the "Object" in Video Object Segmentationcited by 48
  3. 2022Retrospectives on the Embodied AI Workshopcited by 46
  4. 2023Learning Fine-grained View-Invariant Representations from Unpaired Ego-Exo Videos via Temporal Alignmentcited by 46
  5. 2023Panoptic Video Scene Graph Generationcited by 46
  6. 2024Hamba: Single-view 3D Hand Reconstruction with Graph-guided Bi-Scanning Mambacited by 45
  7. 2024LVOS: A Benchmark for Large-scale Long-term Video Object Segmentationcited by 41
  8. 2024Towards Generalist Robot Learning from Internet Video: A Surveycited by 40
  9. 2024HRP: Human Affordances for Robotic Pre-Trainingcited by 40
  10. 2025DICEPTION: A Generalist Diffusion Model for Visual Perceptual Taskscited by 37
  11. 2023Time Does Tell: Self-Supervised Time-Tuning of Dense Image Representationscited by 35
  12. 2024AMEGO: Active Memory from long EGOcentric videoscited by 35
  13. 20233D Hand Pose Estimation in Everyday Egocentric Imagescited by 33
  14. 2024GigaHands: A Massive Annotated Dataset of Bimanual Hand Activitiescited by 32
  15. 2023Helping Hands: An Object-Aware Ego-Centric Video Recognition Modelcited by 31
  16. 2023LEGO: Learning EGOcentric Action Frame Generation via Visual Instruction Tuningcited by 28
  17. 2024Spatial Cognition from Egocentric Video: Out of Sight, Not Out of Mindcited by 28
  18. 2023EgoTracks: A Long-term Egocentric Visual Object Tracking Datasetcited by 27
  19. 2023Generating Visual Scenes from Touchcited by 26
  20. 2024HO-Cap: A Capture System and Dataset for 3D Reconstruction and Pose Tracking of Hand-Object Interactioncited by 25
  21. 2023What a MESS: Multi-Domain Evaluation of Zero-Shot Semantic Segmentationcited by 24
  22. 2023Opening the Vocabulary of Egocentric Actionscited by 23
  23. 2024RMem: Restricted Memory Banks Improve Video Object Segmentationcited by 22
  24. 2024EgoGaussian: Dynamic Scene Understanding from Egocentric Video with 3D Gaussian Splattingcited by 22
  25. 2025EgoExoBench: A Benchmark for First- and Third-person View Video Understanding in MLLMscited by 22
  26. 2023Summarize the Past to Predict the Future: Natural Language Descriptions of Context Boost Multimodal Object Interaction Anticipationcited by 21
  27. 2023Look Ma, No Hands! Agent-Environment Factorization of Egocentric Videoscited by 21
  28. 2024Diff-IP2D: Diffusion-Based Hand-Object Interaction Prediction on Egocentric Videoscited by 21
  29. 2025EdgeTAM: On-Device Track Anything Modelcited by 21
  30. 2024Distilling Vision-Language Models on Millions of Videoscited by 20
  31. 2025Challenges and Trends in Egocentric Vision: A Surveycited by 20
  32. 2023Learning the What and How of Annotation in Video Object Segmentationcited by 19
  33. 2025EgoExo-Gen: Ego-centric Video Prediction by Watching Exo-centric Videoscited by 19
  34. 2023Multi-label affordance mapping from egocentric visioncited by 18
  35. 2024MADiff: Motion-Aware Mamba Diffusion Models for Hand Trajectory Prediction on Egocentric Videoscited by 17
  36. 2024EAGLE: Egocentric AGgregated Language-video Enginecited by 16
  37. 2025The Invisible EgoHand: 3D Hand Forecasting through EgoBody Pose Estimationcited by 16
  38. 2024ZIM: Zero-Shot Image Matting for Anythingcited by 15

50 most cited of 139 All 139 (JSON)