# Look-into-Object: Self-supervised Structure Modeling for Object Recognition

arXiv: https://arxiv.org/abs/2003.14142 · HTML: https://mlsanity.com/paper/2003.14142/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Mar 2020.

## Built on by (most cited first, 20 of 20)

- [Dual Cross-Attention Learning for Fine-Grained Visual Categorization and Object Re-Identification](https://mlsanity.com/paper/2205.02151/) (2022)
- [Joint Representation Learning and Novel Category Discovery on Single- and Multi-modal Data](https://mlsanity.com/paper/2104.12673/) (2021)
- [Fine-Grained Object Classification via Self-Supervised Pose Alignment](https://mlsanity.com/paper/2203.15987/) (2022)
- [SR-GNN: Spatial Relation-aware Graph Neural Network for Fine-Grained Image Categorization](https://mlsanity.com/paper/2209.02109/) (2022)
- [SIM-Trans: Structure Information Modeling Transformer for Fine-grained Visual Categorization](https://mlsanity.com/paper/2208.14607/) (2022)
- [Fine-Grained Visual Classification using Self Assessment Classifier](https://mlsanity.com/paper/2205.10529/) (2022)
- [Enabling Resource-efficient AIoT System with Cross-level Optimization: A survey](https://mlsanity.com/paper/2309.15467/) (2023)
- [Learning Canonical 3D Object Representation for Fine-Grained Recognition](https://mlsanity.com/paper/2108.04628/) (2021)
- [Convolutional Fine-Grained Classification with Self-Supervised Target Relation Regularization](https://mlsanity.com/paper/2208.01997/) (2022)
- [Spatial-Temporal Graph Mamba for Music-Guided Dance Video Synthesis](https://mlsanity.com/paper/2507.06689/) (2025)
- [Feature Boosting, Suppression, and Diversification for Fine-Grained Visual Classification](https://mlsanity.com/paper/2103.02782/) (2021)
- [Parsing Objects at a Finer Granularity: A Survey](https://mlsanity.com/paper/2212.13693/) (2022)
- [How PARTs assemble into wholes: Learning the relative composition of images](https://mlsanity.com/paper/2506.03682/) (2025)
- [CEKD:Cross Ensemble Knowledge Distillation for Augmented Fine-grained Data](https://mlsanity.com/paper/2203.06551/) (2022)
- [GPPF: A General Perception Pre-training Framework via Sparsely Activated Multi-Task Learning](https://mlsanity.com/paper/2208.02148/) (2022)
- [Progressive Multi-task Anti-Noise Learning and Distilling Frameworks for Fine-grained Vehicle Recognition](https://mlsanity.com/paper/2401.14336/) (2024)
- [LoDisc: Learning Global-Local Discriminative Features for Self-Supervised Fine-Grained Visual Recognition](https://mlsanity.com/paper/2403.04066/) (2024)
- [Spatial Cross-Attention Improves Self-Supervised Visual Representation Learning](https://mlsanity.com/paper/2206.05028/) (2022)
- [Semantic Feature Integration network for Fine-grained Visual Classification](https://mlsanity.com/paper/2302.10275/) (2023)
- [Improving Deep Representation Learning via Auxiliary Learnable Target Coding](https://mlsanity.com/paper/2305.18680/) (2023)

## Abstract (arXiv)

Most object recognition approaches predominantly focus on learning discriminative visual patterns while overlooking the holistic object structure. Though important, structure modeling usually requires significant manual annotations and therefore is labor-intensive. In this paper, we propose to "look into object" (explicitly yet intrinsically model the object structure) through incorporating self-supervisions into the traditional framework. We show the recognition backbone can be substantially enhanced for more robust representation learning, without any cost of extra annotation and inference speed. Specifically, we first propose an object-extent learning module for localizing the object according to the visual patterns shared among the instances in the same category. We then design a spatial context learning module for modeling the internal structures of the object, through predicting the relative positions within the extent. These two modules can be easily plugged into any backbone networks during training and detached at inference time. Extensive experiments show that our look-into-object approach (LIO) achieves large performance gain on a number of benchmarks, including generic object recognition (ImageNet) and fine-grained object recognition tasks (CUB, Cars, Aircraft). We also show that this learning paradigm is highly generalizable to other tasks such as object detection and segmentation (MS COCO). Project page: https://github.com/JDAI-CV/LIO.