# Multi-direction and Multi-scale Pyramid in Transformer for Video-based Pedestrian Retrieval

arXiv: https://arxiv.org/abs/2202.06014 · HTML: https://mlsanity.com/paper/2202.06014/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 36 references that are in the map.
- First posted on arXiv 12 Feb 2022.

## Builds on (most cited first, 18 of 18)

- [Video Swin Transformer](https://mlsanity.com/paper/2106.13230/) (2021)
- [Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition](https://mlsanity.com/paper/1406.4729/) (2014)
- [Deep Learning for Person Re-identification: A Survey and Outlook](https://mlsanity.com/paper/2001.04193/) (2020)
- [Beyond Part Models: Person Retrieval with Refined Part Pooling (and a Strong Convolutional Baseline)](https://mlsanity.com/paper/1711.09349/) (2017)
- [Horizontal Pyramid Matching for Person Re-identification](https://mlsanity.com/paper/1804.05275/) (2018)
- [Appearance-Preserving 3D Convolution for Video-based Person Re-identification](https://mlsanity.com/paper/2007.08434/) (2020)
- [Learning Multi-Granular Hypergraphs for Video-Based Person Re-Identification](https://mlsanity.com/paper/2104.14913/) (2021)
- [VRSTC: Occlusion-Free Video Person Re-Identification](https://mlsanity.com/paper/1907.08427/) (2019)
- [Temporal Complementary Learning for Video Person Re-Identification](https://mlsanity.com/paper/2007.09357/) (2020)
- [Watching You: Global-guided Reciprocal Learning for Video-based Person Re-identification](https://mlsanity.com/paper/2103.04337/) (2021)
- [BiCnet-TKS: Learning Efficient Spatial-Temporal Representation for Video Person Re-Identification](https://mlsanity.com/paper/2104.14783/) (2021)
- [Multi-Granularity Reference-Aided Attentive Feature Aggregation for Video-based Person Re-identification](https://mlsanity.com/paper/2003.12224/) (2020)
- [Robust Re-Identification by Multiple Views Knowledge Distillation](https://mlsanity.com/paper/2007.04174/) (2020)
- [Spatial-Temporal Correlation and Topology Learning for Person Re-Identification in Videos](https://mlsanity.com/paper/2104.08241/) (2021)
- [Learning to Disentangle Scenes for Person Re-identification](https://mlsanity.com/paper/2111.05476/) (2021)
- [Rethinking Temporal Fusion for Video-based Person Re-identification on Semantic and Time Aspect](https://mlsanity.com/paper/1911.12512/) (2019)
- [Co-Saliency Spatio-Temporal Interaction Network for Person Re-Identification in Videos](https://mlsanity.com/paper/2004.04979/) (2020)
- [Exploiting Robust Unsupervised Video Person Re-identification](https://mlsanity.com/paper/2111.05170/) (2021)

## Built on by (most cited first, 17 of 17)

- [BigGait: Learning Gait Representation You Want by Large Vision Models](https://mlsanity.com/paper/2402.19122/) (2024)
- [TF-CLIP: Learning Text-free CLIP for Video-based Person Re-Identification](https://mlsanity.com/paper/2312.09627/) (2023)
- [Cross-Platform Video Person ReID: A New Benchmark Dataset and Adaptation Approach](https://mlsanity.com/paper/2408.07500/) (2024)
- [MEVID: Multi-view Extended Videos with Identities for Video Person Re-Identification](https://mlsanity.com/paper/2211.04656/) (2022)
- [On Denoising Walking Videos for Gait Recognition](https://mlsanity.com/paper/2505.18582/) (2025)
- [ShARc: Shape and Appearance Recognition for Person Identification In-the-wild](https://mlsanity.com/paper/2310.15946/) (2023)
- [Graph Convolution Based Efficient Re-Ranking for Visual Retrieval](https://mlsanity.com/paper/2306.08792/) (2023)
- [HAMoBE: Hierarchical and Adaptive Mixture of Biometric Experts for Video-based Person ReID](https://mlsanity.com/paper/2508.05038/) (2025)
- [Other Tokens Matter: Exploring Global and Local Features of Vision Transformers for Object Re-Identification](https://mlsanity.com/paper/2404.14985/) (2024)
- [OpenAnimals: Revisiting Person Re-Identification for Animals Towards Better Generalization](https://mlsanity.com/paper/2410.00204/) (2024)
- [Coarse Attribute Prediction with Task Agnostic Distillation for Real World Clothes Changing ReID](https://mlsanity.com/paper/2505.12580/) (2025)
- [DisenQ: Disentangling Q-Former for Activity-Biometrics](https://mlsanity.com/paper/2507.07262/) (2025)
- [Silhouette-based Gait Foundation Model](https://mlsanity.com/paper/2512.00691/) (2025)
- [Person detection and re-identification in open-world settings of retail stores and public spaces](https://mlsanity.com/paper/2505.00772/) (2025)
- [IDSelect: A RL-Based Cost-Aware Selection Agent for Video-based Multi-Modal Person Recognition](https://mlsanity.com/paper/2602.18990/) (2026)
- [KeyRe-ID: Keypoint-Guided Person Re-Identification using Part-Aware Representation in Videos](https://mlsanity.com/paper/2507.07393/) (2025)
- [Beyond Pedestrians: Caption-Guided CLIP Framework for High-Difficulty Video-based Person Re-Identification](https://mlsanity.com/paper/2604.07740/) (2026)

## Abstract (arXiv)

In video surveillance, pedestrian retrieval (also called person re-identification) is a critical task. This task aims to retrieve the pedestrian of interest from non-overlapping cameras. Recently, transformer-based models have achieved significant progress for this task. However, these models still suffer from ignoring fine-grained, part-informed information. This paper proposes a multi-direction and multi-scale Pyramid in Transformer (PiT) to solve this problem. In transformer-based architecture, each pedestrian image is split into many patches. Then, these patches are fed to transformer layers to obtain the feature representation of this image. To explore the fine-grained information, this paper proposes to apply vertical division and horizontal division on these patches to generate different-direction human parts. These parts provide more fine-grained information. To fuse multi-scale feature representation, this paper presents a pyramid structure containing global-level information and many pieces of local-level information from different scales. The feature pyramids of all the pedestrian images from the same video are fused to form the final multi-direction and multi-scale feature representation. Experimental results on two challenging video-based benchmarks, MARS and iLIDS-VID, show the proposed PiT achieves state-of-the-art performance. Extensive ablation studies demonstrate the superiority of the proposed pyramid structure. The code is available at https://git.openi.org.cn/zangxh/PiT.git.