# Detect What You Can: Detecting and Representing Objects using Holistic Models and Body Parts

arXiv: https://arxiv.org/abs/1406.2031 · HTML: https://mlsanity.com/paper/1406.2031/

## Key facts

- Cited by 232 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Jun 2014.

## Built on by (most cited first, 30 of 232)

- [Token Merging: Your ViT But Faster](https://mlsanity.com/paper/2210.09461/) (2022)
- [LISA: Reasoning Segmentation via Large Language Model](https://mlsanity.com/paper/2308.00692/) (2023)
- [Deep ViT Features as Dense Visual Descriptors](https://mlsanity.com/paper/2112.05814/) (2021)
- [Semantic-SAM: Segment and Recognize Anything at Any Granularity](https://mlsanity.com/paper/2307.04767/) (2023)
- [Osprey: Pixel Understanding with Visual Instruction Tuning](https://mlsanity.com/paper/2312.10032/) (2023)
- [GSVA: Generalized Segmentation via Multimodal Large Language Models](https://mlsanity.com/paper/2312.10103/) (2023)
- [PACO: Parts and Attributes of Common Objects](https://mlsanity.com/paper/2301.01795/) (2023)
- [Matcher: Segment Anything with One Shot Using All-Purpose Feature Matching](https://mlsanity.com/paper/2305.13310/) (2023)
- [SemMAE: Semantic-Guided Masking for Learning Masked Autoencoders](https://mlsanity.com/paper/2206.10207/) (2022)
- [PartImageNet: A Large, High-Quality Dataset of Parts](https://mlsanity.com/paper/2112.00933/) (2021)
- [DNA-Rendering: A Diverse Neural Actor Repository for High-Fidelity Human-centric Rendering](https://mlsanity.com/paper/2307.10173/) (2023)
- [PSALM: Pixelwise SegmentAtion with Large Multi-Modal Model](https://mlsanity.com/paper/2403.14598/) (2024)
- [One Token to Seg Them All: Language Instructed Reasoning Segmentation in Videos](https://mlsanity.com/paper/2409.19603/) (2024)
- [InvPT: Inverted Pyramid Multi-task Transformer for Dense Scene Understanding](https://mlsanity.com/paper/2203.07997/) (2022)
- [Repurposing GANs for One-shot Semantic Part Segmentation](https://mlsanity.com/paper/2103.04379/) (2021)
- [Fine-Grained Visual Prompting](https://mlsanity.com/paper/2306.04356/) (2023)
- [Exploring Relational Context for Multi-Task Dense Prediction](https://mlsanity.com/paper/2104.13874/) (2021)
- [SAM4MLLM: Enhance Multi-Modal Large Language Model for Referring Expression Segmentation](https://mlsanity.com/paper/2409.10542/) (2024)
- [LASSIE: Learning Articulated Shapes from Sparse Image Ensemble via 3D Part Discovery](https://mlsanity.com/paper/2207.03434/) (2022)
- [EVF-SAM: Early Vision-Language Fusion for Text-Prompted Segment Anything Model](https://mlsanity.com/paper/2406.20076/) (2024)
- [Code-as-Monitor: Constraint-aware Visual Programming for Reactive and Proactive Robotic Failure Detection](https://mlsanity.com/paper/2412.04455/) (2024)
- [Concept Decomposition for Visual Exploration and Inspiration](https://mlsanity.com/paper/2305.18203/) (2023)
- [Factorizing Knowledge in Neural Networks](https://mlsanity.com/paper/2207.03337/) (2022)
- [Multi-Task Dense Prediction via Mixture of Low-Rank Experts](https://mlsanity.com/paper/2403.17749/) (2024)
- [Unsupervised Part Discovery from Contrastive Reconstruction](https://mlsanity.com/paper/2111.06349/) (2021)
- [Part-aware Panoptic Segmentation](https://mlsanity.com/paper/2106.06351/) (2021)
- [OPD: Single-view 3D Openable Part Detection](https://mlsanity.com/paper/2203.16421/) (2022)
- [4D-DRESS: A 4D Dataset of Real-world Human Clothing with Semantic Annotations](https://mlsanity.com/paper/2404.18630/) (2024)
- [Visual Agents as Fast and Slow Thinkers](https://mlsanity.com/paper/2408.08862/) (2024)
- [Going Denser with Open-Vocabulary Part Segmentation](https://mlsanity.com/paper/2305.11173/) (2023)

## Abstract (arXiv)

Detecting objects becomes difficult when we need to deal with large shape deformation, occlusion and low resolution. We propose a novel approach to i) handle large deformations and partial occlusions in animals (as examples of highly deformable objects), ii) describe them in terms of body parts, and iii) detect them when their body parts are hard to detect (e.g., animals depicted at low resolution). We represent the holistic object and body parts separately and use a fully connected model to arrange templates for the holistic object and body parts. Our model automatically decouples the holistic object or body parts from the model when they are hard to detect. This enables us to represent a large number of holistic object and body part combinations to better deal with different "detectability" patterns caused by deformations, occlusion and/or low resolution. We apply our method to the six animal categories in the PASCAL VOC dataset and show that our method significantly improves state-of-the-art (by 4.1% AP) and provides a richer representation for objects. During training we use annotations for body parts (e.g., head, torso, etc), making use of a new dataset of fully annotated object parts for PASCAL VOC 2010, which provides a mask for each part.