# POAR: Towards Open Vocabulary Pedestrian Attribute Recognition

arXiv: https://arxiv.org/abs/2303.14643 · HTML: https://mlsanity.com/paper/2303.14643/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 48 references that are in the map.
- First posted on arXiv 26 Mar 2023.

## Builds on (most cited first, 21 of 21)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Open-vocabulary Object Detection via Vision and Language Knowledge Distillation](https://mlsanity.com/paper/2104.13921/) (2021)
- [TransReID: Transformer-based Object Re-Identification](https://mlsanity.com/paper/2102.04378/) (2021)
- [Open-Set Recognition: a Good Closed-Set Classifier is All You Need?](https://mlsanity.com/paper/2110.06207/) (2021)
- [Learning to Prompt for Open-Vocabulary Object Detection with Vision-Language Model](https://mlsanity.com/paper/2203.14940/) (2022)
- [C2AE: Class Conditioned Auto-Encoder for Open-set Recognition](https://mlsanity.com/paper/1904.01198/) (2019)
- [A Comprehensive Survey of Scene Graphs: Generation and Application](https://mlsanity.com/paper/2104.01111/) (2021)
- [Semantically Self-Aligned Network for Text-to-Image Part-aware Person Re-identification](https://mlsanity.com/paper/2107.12666/) (2021)
- [HydraPlus-Net: Attentive Deep Features for Pedestrian Analysis](https://mlsanity.com/paper/1709.09930/) (2017)
- [Object Detection with a Unified Label Space from Multiple Datasets](https://mlsanity.com/paper/2008.06614/) (2020)
- [Deep Imbalanced Attribute Classification using Visual Attention Aggregation](https://mlsanity.com/paper/1807.03903/) (2018)
- [Pedestrian Attribute Recognition: A Survey](https://mlsanity.com/paper/1901.07474/) (2019)
- [Improving Pedestrian Attribute Recognition With Weakly-Supervised Multi-Scale Attribute-Specific Localization](https://mlsanity.com/paper/1910.04562/) (2019)
- [Spatial and Semantic Consistency Regularizations for Pedestrian Attribute Recognition](https://mlsanity.com/paper/2109.05686/) (2021)
- [Label2Label: A Language Modeling Framework for Multi-Attribute Learning](https://mlsanity.com/paper/2207.08677/) (2022)
- [Localization Guided Learning for Pedestrian Attribute Recognition](https://mlsanity.com/paper/1808.09102/) (2018)
- [Improving Facial Attribute Prediction using Semantic Segmentation](https://mlsanity.com/paper/1704.08740/) (2017)
- [Hierarchical Feature Embedding for Attribute Recognition](https://mlsanity.com/paper/2005.11576/) (2020)
- [Deep View-Sensitive Pedestrian Attribute Inference in an end-to-end Model](https://mlsanity.com/paper/1707.06089/) (2017)
- [Multi-Attribute Open Set Recognition](https://mlsanity.com/paper/2208.06809/) (2022)

## Built on by (most cited first, 3 of 3)

- [FOCUS: Fine-grained Optimization with Semantic Guided Understanding for Pedestrian Attributes Recognition](https://mlsanity.com/paper/2506.22836/) (2025)
- [Object Retrieval for Visual Question Answering with Outside Knowledge](https://mlsanity.com/paper/2403.10798/) (2024)
- [Open-Attribute Person Retrieval: Finding People Through Distinctive and Novel Attributes](https://mlsanity.com/paper/2508.01389/) (2025)

## Abstract (arXiv)

Pedestrian attribute recognition (PAR) aims to predict the attributes of a target pedestrian in a surveillance system. Existing methods address the PAR problem by training a multi-label classifier with predefined attribute classes. However, it is impossible to exhaust all pedestrian attributes in the real world. To tackle this problem, we develop a novel pedestrian open-attribute recognition (POAR) framework. Our key idea is to formulate the POAR problem as an image-text search problem. We design a Transformer-based image encoder with a masking strategy. A set of attribute tokens are introduced to focus on specific pedestrian parts (e.g., head, upper body, lower body, feet, etc.) and encode corresponding attributes into visual embeddings. Each attribute category is described as a natural language sentence and encoded by the text encoder. Then, we compute the similarity between the visual and text embeddings of attributes to find the best attribute descriptions for the input images. Different from existing methods that learn a specific classifier for each attribute category, we model the pedestrian at a part-level and explore the searching method to handle the unseen attributes. Finally, a many-to-many contrastive (MTMC) loss with masked tokens is proposed to train the network since a pedestrian image can comprise multiple attributes. Extensive experiments have been conducted on benchmark PAR datasets with an open-attribute setting. The results verified the effectiveness of the proposed POAR method, which can form a strong baseline for the POAR task. Our code is available at urlhttps://github.com/IvyYZ/POAR.