# ATTACH Dataset: Annotated Two-Handed Assembly Actions for Human Action Understanding arXiv: https://arxiv.org/abs/2304.08210 ยท HTML: https://mlsanity.com/paper/2304.08210/ ## Key facts - Cited by 14 papers in the map (data through 14 Oct 2026). - Builds on 20 of its 30 references that are in the map. - First posted on arXiv 17 Apr 2023. ## Builds on (most cited first, 20 of 20) - [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017) - [Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition](https://mlsanity.com/paper/1801.07455/) (2018) - [Video Swin Transformer](https://mlsanity.com/paper/2106.13230/) (2021) - [HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clips](https://mlsanity.com/paper/1906.03327/) (2019) - [Scaling Egocentric Vision: The EPIC-KITCHENS Dataset](https://mlsanity.com/paper/1804.02748/) (2018) - [NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis](https://mlsanity.com/paper/1604.02808/) (2016) - [Towards Automatic Learning of Procedures from Web Instructional Videos](https://mlsanity.com/paper/1703.09788/) (2017) - [Rescaling Egocentric Vision](https://mlsanity.com/paper/2006.13256/) (2020) - [Hollywood in Homes: Crowdsourcing Data Collection for Activity Understanding](https://mlsanity.com/paper/1604.01753/) (2016) - [NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understanding](https://mlsanity.com/paper/1905.04757/) (2019) - [COIN: A Large-scale Dataset for Comprehensive Instructional Video Analysis](https://mlsanity.com/paper/1903.02874/) (2019) - [Assembly101: A Large-Scale Multi-View Video Dataset for Understanding Procedural Activities](https://mlsanity.com/paper/2203.14712/) (2022) - [Temporal Pyramid Network for Action Recognition](https://mlsanity.com/paper/2004.03548/) (2020) - [Charades-Ego: A Large-Scale Dataset of Paired Third and First Person Videos](https://mlsanity.com/paper/1804.09626/) (2018) - [The IKEA ASM Dataset: Understanding People Assembling Furniture through Actions, Objects and Pose](https://mlsanity.com/paper/2007.00394/) (2020) - [A Short Note on the Kinetics-700-2020 Human Action Dataset](https://mlsanity.com/paper/2010.10864/) (2020) - [View Adaptive Neural Networks for High Performance Skeleton-based Human Action Recognition](https://mlsanity.com/paper/1804.07453/) (2018) - [The MECCANO Dataset: Understanding Human-Object Interactions from Egocentric Videos in an Industrial-like Domain](https://mlsanity.com/paper/2010.05654/) (2020) - [Toyota Smarthome Untrimmed: Real-World Untrimmed Videos for Activity Detection](https://mlsanity.com/paper/2010.14982/) (2020) - [Human Pose Forecasting via Deep Markov Models](https://mlsanity.com/paper/1707.09240/) (2017) ## Built on by (most cited first, 14 of 14) - [IndustReal: A Dataset for Procedure Step Recognition Handling Execution Errors in Egocentric Videos in an Industrial-Like Setting](https://mlsanity.com/paper/2310.17323/) (2023) - [How Object Information Improves Skeleton-based Human Action Recognition in Assembly Tasks](https://mlsanity.com/paper/2306.05844/) (2023) - [PhysLab: A Benchmark Dataset for Multi-Granularity Visual Parsing of Physics Experiments](https://mlsanity.com/paper/2506.06631/) (2025) - [Fusing Hand and Body Skeletons for Human Action Recognition in Assembly](https://mlsanity.com/paper/2307.09238/) (2023) - [A vision-based framework for human behavior understanding in industrial assembly lines](https://mlsanity.com/paper/2409.17356/) (2024) - [AssemblyHands-X: Modeling 3D Hand-Body Coordination for Understanding Bimanual Human Activities](https://mlsanity.com/paper/2509.23888/) (2025) - [Polyphony: Diffusion-based Dual-Hand Action Segmentation with Alternating Vision Transformer and Semantic Conditioning](https://mlsanity.com/paper/2605.31115/) (2026) - [Including Semantic Information via Word Embeddings for Skeleton-based Action Recognition](https://mlsanity.com/paper/2506.18721/) (2025) - [WS2: Weakly Supervised Segmentation using Before-After Supervision in Waste Sorting](https://mlsanity.com/paper/2509.06485/) (2025) - [Vision-Based Mistake Analysis in Procedural Activities: A Review of Advances and Challenges](https://mlsanity.com/paper/2510.19292/) (2025) - [A Synthetic-Driven Vision System for Assembly Step Recognition](https://mlsanity.com/paper/2607.00129/) (2026) - [Compositional Context Fine-Tuning Vision-Language Model for Complex Assembly Action Understanding from Videos](https://mlsanity.com/paper/2607.10797/) (2026) - [Lag-aware cross-hand alignment for dual-hand action segmentation](https://mlsanity.com/paper/2607.26215/) (2026) - [Hand-Aware Transition Modeling for Bimanual Procedural Anomaly Detection](https://mlsanity.com/paper/2609.21207/) (2026) ## Abstract (arXiv) With the emergence of collaborative robots (cobots), human-robot collaboration in industrial manufacturing is coming into focus. For a cobot to act autonomously and as an assistant, it must understand human actions during assembly. To effectively train models for this task, a dataset containing suitable assembly actions in a realistic setting is crucial. For this purpose, we present the ATTACH dataset, which contains 51.6 hours of assembly with 95.2k annotated fine-grained actions monitored by three cameras, which represent potential viewpoints of a cobot. Since in an assembly context workers tend to perform different actions simultaneously with their two hands, we annotated the performed actions for each hand separately. Therefore, in the ATTACH dataset, more than 68% of annotations overlap with other annotations, which is many times more than in related datasets, typically featuring more simplistic assembly tasks. For better generalization with respect to the background of the working area, we did not only record color and depth images, but also used the Azure Kinect body tracking SDK for estimating 3D skeletons of the worker. To create a first baseline, we report the performance of state-of-the-art methods for action recognition as well as action detection on video and skeleton-sequence inputs. The dataset is available at https://www.tu-ilmenau.de/neurob/data-sets-code/attach-dataset .