mlsanityData to 7 Oct 2026

Egocentric Prediction of Action Target in 3D

arXiv 2203.13116 ↗First posted 24 Mar 2022cs.CVcited by 31

Not on the map. Why?

Abstract (arXiv)

We are interested in anticipating as early as possible the target location of a person's object manipulation action in a 3D workspace from egocentric vision. It is important in fields like human-robot collaboration, but has not yet received enough attention from vision and learning communities. To stimulate more research on this challenging egocentric vision task, we propose a large multimodality dataset of more than 1 million frames of RGB-D and IMU streams, and provide evaluation metrics based on our high-quality 2D and 3D labels from semi-automatic annotation. Meanwhile, we design baseline methods using recurrent neural networks and conduct various ablation studies to validate their effectiveness. Our results demonstrate that this new task is worthy of further study by researchers in robotics, vision, and learning communities.

Key facts. Cited by 31 papers in the map (data through 7 Oct 2026). Builds on 27 of its 63 references that are in the map. First posted on arXiv 24 Mar 2022.

Builds on

27 of 63 references.

References · 27

  1. 2015SSD: Single Shot MultiBox Detectorcited by 3,125
  2. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,530
  3. 2018Scaling Egocentric Vision: The EPIC-KITCHENS Datasetcited by 789
  4. 2018PointConv: Deep Convolutional Networks on 3D Point Cloudscited by 656
  5. 2020Understanding Human Hands in Contact at Internet Scalecited by 281
  6. 2017First-Person Hand Action Benchmark with RGB-D Videos and 3D Hand Pose Annotationscited by 229
  7. 2019EPIC-Fusion: Audio-Visual Temporal Binding for Egocentric Action Recognitioncited by 226
  8. 2020The EPIC-KITCHENS Dataset: Collection, Challenges and Baselinescited by 221
  9. 2021Anticipative Video Transformercited by 197
  10. 2018Grounded Human-Object Interaction Hotspots from Videocited by 128
  11. 2020Rolling-Unrolling LSTMs for Action Anticipation from First-Person Videocited by 119
  12. 2019What Would You Expect? Anticipating Egocentric Actions with Rolling-Unrolling LSTMs and Modality Attentioncited by 104
15 more
  1. 2021Ego-Exo: Transferring Visual Representations from Third-person to First-person Videoscited by 98
  2. 2019You2Me: Inferring Body Pose in Egocentric Video via First and Second Person Interactionscited by 82
  3. 2020The MECCANO Dataset: Understanding Human-Object Interactions from Egocentric Videos in an Industrial-like Domaincited by 79
  4. 2020In the Eye of the Beholder: Gaze and Actions in First Person Videocited by 68
  5. 2018LSTA: Long Short-Term Attention for Egocentric Action Recognitioncited by 57
  6. 2019Predicting the Future: A Jointly Learnt Model for Action Anticipationcited by 41
  7. 2021Predicting the Future from First Person (Egocentric) Vision: A Surveycited by 36
  8. 2019Analysis of the hands in egocentric vision: A surveycited by 25
  9. 2015Predicting Important Objects for Egocentric Video Summarizationcited by 21
  10. 2021Is First Person Vision Challenging for Object Tracking?cited by 21
  11. 2016First Person Action-Object Detection with EgoNetcited by 17
  12. 2018Personalized Dynamics Models for Adaptive Assistive Navigation Systemscited by 13
  13. 2017Anticipating Daily Intention using On-Wrist Motion Triggered Sensingcited by 8
  14. 2018Egocentric Basketball Motion Planning from a Single First-Person Imagecited by 8
  15. 2016Prediction of Manipulation Actionscited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 4; 2025: 13; 2026: 10.

31 papers.

Sort:

By area:

  1. 2025EgoLife: Towards Egocentric Life Assistantcited by 86
  2. 2023An Outlook into the Future of Egocentric Visioncited by 84
  3. 2025Vision-Language-Action Models for Robotics: A Review Towards Real-World Applicationscited by 82
  4. 2024General Flow as Foundation Affordance for Scalable Robot Learningcited by 80
  5. 2025villa-X: Enhancing Latent Action Modeling in Vision-Language-Action Modelscited by 58
  6. 2026World Action Models: The Next Frontier in Embodied AIcited by 46
  7. 2023Uncertainty-aware State Space Transformer for Egocentric 3D Hand Trajectory Forecastingcited by 33
  8. 2023EgoDistill: Egocentric Head Motion Distillation for Efficient Video Understandingcited by 32
  9. 2024Diff-IP2D: Diffusion-Based Hand-Object Interaction Prediction on Egocentric Videoscited by 21
  10. 2023Instance Tracking in 3D Scenes from Egocentric Videoscited by 17
  11. 2024MADiff: Motion-Aware Mamba Diffusion Models for Hand Trajectory Prediction on Egocentric Videoscited by 17
  12. 2025The Invisible EgoHand: 3D Hand Forecasting through EgoBody Pose Estimationcited by 16
19 more
  1. 2025Co-Evolving Latent Action World Modelscited by 12
  2. 2025Generating 6DoF Object Manipulation Trajectories from Action Description in Egocentric Visioncited by 11
  3. 2026World Action Models: A Surveycited by 10
  4. 2024EMAG: Ego-motion Aware and Generalizable 2D Hand Forecasting from Egocentric Videoscited by 8
  5. 2025Novel Diffusion Models for Multimodal 3D Hand Trajectory Predictioncited by 7
  6. 2025Flowing from Reasoning to Motion: Learning 3D Hand Trajectory Prediction from Egocentric Human Interaction Videoscited by 6
  7. 2026Learning Additively Compositional Latent Actions for Embodied AIcited by 6
  8. 2026SpaMEM: Benchmarking Dynamic Spatial Reasoning via Perception-Memory Integration in Embodied Environmentscited by 6
  9. 2025EgoAdapt: Adaptive Multisensory Distillation and Policy Learning for Efficient Egocentric Perceptioncited by 4
  10. 2025Uni-Hand: Universal Hand Motion Forecasting in Egocentric Viewscited by 4
  11. 2025Zero-Shot Temporal Interaction Localization for Egocentric Videoscited by 2
  12. 2025Gaze Beyond the Frame: Forecasting Egocentric 3D Visual Spancited by 2
  13. 2026EgoTac: In-the-wild Tactile Prediction from Egocentric Visioncited by 2
  14. 2026Spatially Prompted Visual Trajectory Prediction for Egocentric Manipulationcited by 1
  15. 2025EgoLoc: A Generalizable Solution for Temporal Interaction Localization in Egocentric Videoscited by 0
  16. 2026MobiDiary: Autoregressive Action Captioning with Wearable Devices and Wireless Signalscited by 0
  17. 2026From Where to How: Continuous 4D Interaction Forecasting from Egocentric Videocited by 0
  18. 2026HAP: A Hand-Driven Active Perception Framework for Egocentric Head Motion Predictioncited by 0
  19. 2026AI Smart Glasses for Wearable Intelligence: From Egocentric Sensing to Agentic Personalizationcited by 0