# QUB-PHEO: A Visual-Based Dyadic Multi-View Dataset for Intention Inference in Collaborative Assembly

arXiv: https://arxiv.org/abs/2409.15560 · HTML: https://mlsanity.com/paper/2409.15560/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 74 references that are in the map.
- First posted on arXiv 23 Sep 2024.

## Builds on (most cited first, 9 of 9)

- [Ego4D: Around the World in 3,000 Hours of Egocentric Video](https://mlsanity.com/paper/2110.07058/) (2021)
- [MediaPipe Hands: On-device Real-time Hand Tracking](https://mlsanity.com/paper/2006.10214/) (2020)
- [Context-Aware Personality Inference in Dyadic Scenarios: Introducing the UDIVA Dataset](https://mlsanity.com/paper/2012.14259/) (2020)
- [Nonverbal Cues in Human-Robot Interaction: A Communication Studies Perspective](https://mlsanity.com/paper/2304.11293/) (2023)
- [Developing Social Robots with Empathetic Non-Verbal Cues Using Large Language Models](https://mlsanity.com/paper/2308.16529/) (2023)
- [A Study on the Challenges of Using Robotics Simulators for Testing](https://mlsanity.com/paper/2004.07368/) (2020)
- [Dyadic Speech-based Affect Recognition using DAMI-P2C Parent-child Multimodal Interaction Dataset](https://mlsanity.com/paper/2008.09207/) (2020)
- [Presentation and Analysis of a Multimodal Dataset for Grounded Language Learning](https://mlsanity.com/paper/2007.14987/) (2020)
- [Improving Intention Detection in Single-Trial Classification through Fusion of EEG and Eye-tracker Data](https://mlsanity.com/paper/2112.02566/) (2021)

## Abstract (arXiv)

QUB-PHEO introduces a visual-based, dyadic dataset with the potential of advancing human-robot interaction (HRI) research in assembly operations and intention inference. This dataset captures rich multimodal interactions between two participants, one acting as a 'robot surrogate,' across a variety of assembly tasks that are further broken down into 36 distinct subtasks. With rich visual annotations, such as facial landmarks, gaze, hand movements, object localization, and more for 70 participants, QUB-PHEO offers two versions: full video data for 50 participants and visual cues for all 70. Designed to improve machine learning models for HRI, QUB-PHEO enables deeper analysis of subtle interaction cues and intentions, promising contributions to the field. The dataset will be available at https://github.com/exponentialR/QUB-PHEO subject to an End-User License Agreement (EULA).