# MoGaze: A Dataset of Full-Body Motions that Includes Workspace Geometry and Eye-Gaze

arXiv: https://arxiv.org/abs/2011.11552 · HTML: https://mlsanity.com/paper/2011.11552/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Nov 2020.

## Built on by (most cited first, 21 of 21)

- [GIMO: Gaze-Informed Human Motion Prediction in Context](https://mlsanity.com/paper/2204.09443/) (2022)
- [GRIP: Generating Interaction Poses Using Spatial Cues and Latent Consistency](https://mlsanity.com/paper/2308.11617/) (2023)
- [A Survey on Human Interaction Motion Generation](https://mlsanity.com/paper/2503.12763/) (2025)
- [HOIMotion: Forecasting Human Motion During Human-Object Interactions Using Egocentric 3D Object Bounding Boxes](https://mlsanity.com/paper/2407.02633/) (2024)
- [GazeMoDiff: Gaze-guided Diffusion Model for Stochastic Human Motion Prediction](https://mlsanity.com/paper/2312.12090/) (2023)
- [Pose2Gaze: Eye-body Coordination during Daily Activities for Gaze Prediction from Full-body Poses](https://mlsanity.com/paper/2312.12042/) (2023)
- [3D Human Interaction Generation: A Survey](https://mlsanity.com/paper/2503.13120/) (2025)
- [GazeHTA: End-to-end Gaze Target Detection with Head-Target Association](https://mlsanity.com/paper/2404.10718/) (2024)
- [GazeMotion: Gaze-guided Human Motion Forecasting](https://mlsanity.com/paper/2403.09885/) (2024)
- [Gaze-guided Hand-Object Interaction Synthesis: Dataset and Method](https://mlsanity.com/paper/2403.16169/) (2024)
- [Preemptive Motion Planning for Human-to-Robot Indirect Placement Handovers](https://mlsanity.com/paper/2203.00156/) (2022)
- [Learning Predictive Visuomotor Coordination](https://mlsanity.com/paper/2503.23300/) (2025)
- [Shape-aware Inertial Poser: Motion Tracking for Humans with Diverse Shapes Using Sparse Inertial Sensors](https://mlsanity.com/paper/2510.17101/) (2025)
- [Prime and Reach: Synthesising Body Motion for Gaze-Primed Object Reach](https://mlsanity.com/paper/2512.16456/) (2025)
- [Human locomotor control timescales depend on the environmental context and sensory input modality](https://mlsanity.com/paper/2503.16340/) (2025)
- [Trajectory prediction for heterogeneous agents: A performance analysis on small and imbalanced datasets](https://mlsanity.com/paper/2510.03776/) (2025)
- [DexterCap: An Affordable and Automated System for Capturing Dexterous Hand-Object Manipulation](https://mlsanity.com/paper/2601.05844/) (2026)
- [ObjectVisA-120: Object-based Visual Attention Prediction in Interactive Street-crossing Environments](https://mlsanity.com/paper/2601.13218/) (2026)
- [Int3DNet: Scene-Motion Cross Attention Network for 3D Intention Prediction in Mixed Reality](https://mlsanity.com/paper/2603.13355/) (2026)
- [From Where to How: Continuous 4D Interaction Forecasting from Egocentric Video](https://mlsanity.com/paper/2609.08636/) (2026)
- [Learning from Humans for Proactive Assistance in Human-Robot Collaborative Transport](https://mlsanity.com/paper/2609.25351/) (2026)

## Abstract (arXiv)

As robots become more present in open human environments, it will become crucial for robotic systems to understand and predict human motion. Such capabilities depend heavily on the quality and availability of motion capture data. However, existing datasets of full-body motion rarely include 1) long sequences of manipulation tasks, 2) the 3D model of the workspace geometry, and 3) eye-gaze, which are all important when a robot needs to predict the movements of humans in close proximity. Hence, in this paper, we present a novel dataset of full-body motion for everyday manipulation tasks, which includes the above. The motion data was captured using a traditional motion capture system based on reflective markers. We additionally captured eye-gaze using a wearable pupil-tracking device. As we show in experiments, the dataset can be used for the design and evaluation of full-body motion prediction algorithms. Furthermore, our experiments show eye-gaze as a powerful predictor of human intent. The dataset includes 180 min of motion capture data with 1627 pick and place actions being performed. It is available at https://humans-to-robots-motion.github.io/mogaze and is planned to be extended to collaborative tasks with two humans in the near future.