# Fine-grained activity recognition for assembly videos

arXiv: https://arxiv.org/abs/2012.01392 · HTML: https://mlsanity.com/paper/2012.01392/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Dec 2020.

## Built on by (most cited first, 4 of 4)

- [Fine-grained Activities of People Worldwide](https://mlsanity.com/paper/2207.05182/) (2022)
- [Supervised Representation Learning towards Generalizable Assembly State Recognition](https://mlsanity.com/paper/2408.11700/) (2024)
- [Activity Recognition in Assembly Tasks by Bayesian Filtering in Multi-Hypergraphs](https://mlsanity.com/paper/2202.00332/) (2022)
- [Hand Guided High Resolution Feature Enhancement for Fine-Grained Atomic Action Segmentation within Complex Human Assemblies](https://mlsanity.com/paper/2211.13694/) (2022)

## Abstract (arXiv)

In this paper we address the task of recognizing assembly actions as a structure (e.g. a piece of furniture or a toy block tower) is built up from a set of primitive objects. Recognizing the full range of assembly actions requires perception at a level of spatial detail that has not been attempted in the action recognition literature to date. We extend the fine-grained activity recognition setting to address the task of assembly action recognition in its full generality by unifying assembly actions and kinematic structures within a single framework. We use this framework to develop a general method for recognizing assembly actions from observation sequences, along with observation features that take advantage of a spatial assembly's special structure. Finally, we evaluate our method empirically on two application-driven data sources: (1) An IKEA furniture-assembly dataset, and (2) A block-building dataset. On the first, our system recognizes assembly actions with an average framewise accuracy of 70% and an average normalized edit distance of 10%. On the second, which requires fine-grained geometric reasoning to distinguish between assemblies, our system attains an average normalized edit distance of 23% -- a relative improvement of 69% over prior work.