FT-HID: A Large Scale RGB-D Dataset for First and Third Person Human Interaction Analysis
Not on the map. Why?
Abstract (arXiv)
Analysis of human interaction is one important research topic of human motion analysis. It has been studied either using first person vision (FPV) or third person vision (TPV). However, the joint learning of both types of vision has so far attracted little attention. One of the reasons is the lack of suitable datasets that cover both FPV and TPV. In addition, existing benchmark datasets of either FPV or TPV have several limitations, including the limited number of samples, participant subjects, interaction categories, and modalities. In this work, we contribute a large-scale human interaction dataset, namely, FT-HID dataset. FT-HID contains pair-aligned samples of first person and third person visions. The dataset was collected from 109 distinct subjects and has more than 90K samples for three modalities. The dataset has been validated by using several existing action recognition methods. In addition, we introduce a novel multi-view interaction mechanism for skeleton sequences, and a joint learning multi-stream framework for first person and third person visions. Both methods yield promising results on the FT-HID dataset. It is expected that the introduction of this vision-aligned large-scale dataset will promote the development of both FPV and TPV, and their joint learning techniques for human action analysis. The dataset and code are available at hrefhttps://github.com/ENDLICHERE/FT-HIDhere.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 22 of its 94 references that are in the map. First posted on arXiv 21 Sep 2022.
Builds on
22 of 94 references.
References · 22
- 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
- 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
- 2015Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcastingcited by 1,428
- 2016NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysiscited by 683
- 2019NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understandingcited by 530
- 2016Convolutional Two-Stream Network Fusion for Video Action Recognitioncited by 384
- 2017First-Person Hand Action Benchmark with RGB-D Videos and 3D Hand Pose Annotationscited by 229
- 2019Actional-Structural Graph Convolutional Networks for Skeleton-based Action Recognitioncited by 228
- 2014Cross-view Action Modeling, Learning and Recognitioncited by 134
- 2018View Adaptive Neural Networks for High Performance Skeleton-based Human Action Recognitioncited by 84
- 2018Modality Distillation with Multiple Stream Networks for Action Recognitioncited by 83
- 2015Rank Pooling for Action Recognitioncited by 29
10 more
- 2016Deep Multimodal Feature Analysis for Action Recognition in RGB+D Videoscited by 26
- 2014HOPC: Histogram of Oriented Principal Components of 3D Pointclouds for Action Recognitioncited by 15
- 2017Investigation of Different Skeleton Features for CNN-based 3D Action Recognitioncited by 12
- 2017Cooperative Training of Deep Aggregation Networks for RGB-D Action Recognitioncited by 11
- 2016Action Recognition Based on Joint Trajectory Maps with Convolutional Neural Networkscited by 10
- 2017Generalized Rank Pooling for Activity Recognitioncited by 10
- 2021A Two-stream Neural Network for Pose-based Hand Gesture Recognitioncited by 10
- 2016Egocentric Activity Recognition with Multimodal Fisher Vectorcited by 5
- 2017Viewpoint Invariant Action Recognition using RGB-D Videoscited by 5
- 20153D Human Activity Recognition with Reconfigurable Convolutional Neural Networkscited by 4
Built on by
Citing papers per year: 2025: 2; 2026: 1.
3 papers.