# Aligned to the Object, not to the Image: A Unified Pose-aligned Representation for Fine-grained Recognition

arXiv: https://arxiv.org/abs/1801.09057 · HTML: https://mlsanity.com/paper/1801.09057/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Jan 2018.

## Built on by (most cited first, 10 of 10)

- [Learning Robust Visual-semantic Mapping for Zero-shot Learning](https://mlsanity.com/paper/2104.05668/) (2021)
- [Semi-supervised Keypoint Localization](https://mlsanity.com/paper/2101.07988/) (2021)
- [A Novel Plug-in Module for Fine-Grained Visual Classification](https://mlsanity.com/paper/2202.03822/) (2022)
- [Creative Birds: Self-Supervised Single-View 3D Style Transfer](https://mlsanity.com/paper/2307.14127/) (2023)
- [Learning Canonical 3D Object Representation for Fine-Grained Recognition](https://mlsanity.com/paper/2108.04628/) (2021)
- [Weakly Supervised Keypoint Discovery](https://mlsanity.com/paper/2109.13423/) (2021)
- [M2Former: Multi-Scale Patch Selection for Fine-Grained Visual Recognition](https://mlsanity.com/paper/2308.02161/) (2023)
- [ViT-FOD: A Vision Transformer based Fine-grained Object Discriminator](https://mlsanity.com/paper/2203.12816/) (2022)
- [Multi-scale Activation, Refinement, and Aggregation: Exploring Diverse Cues for Fine-Grained Bird Recognition](https://mlsanity.com/paper/2504.09215/) (2025)
- [LMPNet for Weakly-supervised Keypoint Discovery](https://mlsanity.com/paper/2507.02308/) (2025)

## Abstract (arXiv)

Dramatic appearance variation due to pose constitutes a great challenge in fine-grained recognition, one which recent methods using attention mechanisms or second-order statistics fail to adequately address. Modern CNNs typically lack an explicit understanding of object pose and are instead confused by entangled pose and appearance. In this paper, we propose a unified object representation built from a hierarchy of pose-aligned regions. Rather than representing an object by regions aligned to image axes, the proposed representation characterizes appearance relative to the object's pose using pose-aligned patches whose features are robust to variations in pose, scale and rotation. We propose an algorithm that performs pose estimation and forms the unified object representation as the concatenation of hierarchical pose-aligned regions features, which is then fed into a classification network. The proposed algorithm surpasses the performance of other approaches, increasing the state-of-the-art by nearly 2% on the widely-used CUB-200 dataset and by more than 8% on the much larger NABirds dataset. The effectiveness of this paradigm relative to competing methods suggests the critical importance of disentangling pose and appearance for continued progress in fine-grained recognition.