# Beyond Holistic Object Recognition: Enriching Image Understanding with Part States

arXiv: https://arxiv.org/abs/1612.07310 · HTML: https://mlsanity.com/paper/1612.07310/

## Key facts

- Cited by 13 papers in the map (data through 7 Oct 2026).
- First posted on arXiv 15 Dec 2016.

## Built on by (most cited first, 13 of 13)

- [ShapeLLM: Universal 3D Object Understanding for Embodied Interaction](https://mlsanity.com/paper/2402.17766/) (2024)
- [OPD: Single-view 3D Openable Part Detection](https://mlsanity.com/paper/2203.16421/) (2022)
- [Interactiveness Field in Human-Object Interactions](https://mlsanity.com/paper/2204.07718/) (2022)
- [Mining Cross-Person Cues for Body-Part Interactiveness Learning in HOI Detection](https://mlsanity.com/paper/2207.14192/) (2022)
- [HAKE: A Knowledge Engine Foundation for Human Activity Understanding](https://mlsanity.com/paper/2202.06851/) (2022)
- [From Isolated Islands to Pangea: Unifying Semantic Space for Human Action Understanding](https://mlsanity.com/paper/2304.00553/) (2023)
- [A Fine-Grained Dataset and its Efficient Semantic Segmentation for Unstructured Driving Scenarios](https://mlsanity.com/paper/2103.13109/) (2021)
- [Symbol-LLM: Leverage Language Models for Symbolic System in Visual Human Activity Reasoning](https://mlsanity.com/paper/2311.17365/) (2023)
- [ELLIPSDF: Joint Object Pose and Shape Optimization with a Bi-level Ellipsoid and Signed Distance Function Description](https://mlsanity.com/paper/2108.00355/) (2021)
- [Highlighting Object Category Immunity for the Generalization of Human-Object Interaction Detection](https://mlsanity.com/paper/2202.09492/) (2022)
- [Articulate3D: Holistic Understanding of 3D Scenes as Universal Scene Description](https://mlsanity.com/paper/2412.01398/) (2024)
- [Discovering A Variety of Objects in Spatio-Temporal Human-Object Interactions](https://mlsanity.com/paper/2211.07501/) (2022)
- [Transferable Interactiveness Knowledge for Human-Object Interaction Detection](https://mlsanity.com/paper/2101.10292/) (2021)

## Abstract (arXiv)

Important high-level vision tasks such as human-object interaction, image captioning and robotic manipulation require rich semantic descriptions of objects at part level. Based upon previous work on part localization, in this paper, we address the problem of inferring rich semantics imparted by an object part in still images. We propose to tokenize the semantic space as a discrete set of part states. Our modeling of part state is spatially localized, therefore, we formulate the part state inference problem as a pixel-wise annotation problem. An iterative part-state inference neural network is specifically designed for this task, which is efficient in time and accurate in performance. Extensive experiments demonstrate that the proposed method can effectively predict the semantic states of parts and simultaneously correct localization errors, thus benefiting a few visual understanding applications. The other contribution of this paper is our part state dataset which contains rich part-level semantic annotations.