# Spatio-Temporal Representation Factorization for Video-based Person Re-Identification

arXiv: https://arxiv.org/abs/2107.11878 · HTML: https://mlsanity.com/paper/2107.11878/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 68 references that are in the map.
- First posted on arXiv 25 Jul 2021.

## Builds on (most cited first, 30 of 35)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [SlowFast Networks for Video Recognition](https://mlsanity.com/paper/1812.03982/) (2018)
- [Image Super-Resolution Using Very Deep Residual Channel Attention Networks](https://mlsanity.com/paper/1807.02758/) (2018)
- [Random Erasing Data Augmentation](https://mlsanity.com/paper/1708.04896/) (2017)
- [Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer](https://mlsanity.com/paper/1612.03928/) (2016)
- [In Defense of the Triplet Loss for Person Re-Identification](https://mlsanity.com/paper/1703.07737/) (2017)
- [Performance Measures and a Data Set for Multi-Target, Multi-Camera Tracking](https://mlsanity.com/paper/1609.01775/) (2016)
- [MoCoGAN: Decomposing Motion and Content for Video Generation](https://mlsanity.com/paper/1707.04993/) (2017)
- [Rethinking Spatiotemporal Feature Learning: Speed-Accuracy Trade-offs in Video Classification](https://mlsanity.com/paper/1712.04851/) (2017)
- [Beyond Part Models: Person Retrieval with Refined Part Pooling (and a Strong Convolutional Baseline)](https://mlsanity.com/paper/1711.09349/) (2017)
- [Learning Spatio-Temporal Representation with Pseudo-3D Residual Networks](https://mlsanity.com/paper/1711.10305/) (2017)
- [Harmonious Attention Network for Person Re-Identification](https://mlsanity.com/paper/1802.08122/) (2018)
- [R-C3D: Region Convolutional 3D Network for Temporal Activity Detection](https://mlsanity.com/paper/1703.07814/) (2017)
- [Style Normalization and Restitution for Generalizable Person Re-identification](https://mlsanity.com/paper/2005.11037/) (2020)
- [Relation-Aware Global Attention for Person Re-identification](https://mlsanity.com/paper/1904.02998/) (2019)
- [Dual Attention Networks for Multimodal Reasoning and Matching](https://mlsanity.com/paper/1611.00471/) (2016)
- [Appearance-Preserving 3D Convolution for Video-based Person Re-identification](https://mlsanity.com/paper/2007.08434/) (2020)
- [Learning Multi-Granular Hypergraphs for Video-Based Person Re-Identification](https://mlsanity.com/paper/2104.14913/) (2021)
- [Diversity Regularized Spatiotemporal Attention for Video-based Person Re-identification](https://mlsanity.com/paper/1803.09882/) (2018)
- [VRSTC: Occlusion-Free Video Person Re-Identification](https://mlsanity.com/paper/1907.08427/) (2019)
- [Temporal Complementary Learning for Video Person Re-Identification](https://mlsanity.com/paper/2007.09357/) (2020)
- [Multi-Granularity Reference-Aided Attentive Feature Aggregation for Video-based Person Re-identification](https://mlsanity.com/paper/2003.12224/) (2020)
- [Re-Identification with Consistent Attentive Siamese Networks](https://mlsanity.com/paper/1811.07487/) (2018)
- [Jointly Attentive Spatial-Temporal Pooling Networks for Video-based Person Re-Identification](https://mlsanity.com/paper/1708.02286/) (2017)
- [Revisiting Temporal Modeling for Video-based Person ReID](https://mlsanity.com/paper/1805.02104/) (2018)

## Built on by (most cited first, 29 of 29)

- [Adversarial Attacks on Black Box Video Classifiers: Leveraging the Power of Geometric Transformations](https://mlsanity.com/paper/2110.01823/) (2021)
- [Learning Modal-Invariant and Temporal-Memory for Video-based Visible-Infrared Person Re-Identification](https://mlsanity.com/paper/2208.02450/) (2022)
- [Causal Reasoning Meets Visual Representation Learning: A Prospective Study](https://mlsanity.com/paper/2204.12037/) (2022)
- [Deeply-Coupled Convolution-Transformer with Spatial-temporal Complementary Learning for Video-based Person Re-identification](https://mlsanity.com/paper/2304.14122/) (2023)
- [Cross-Domain Video Anomaly Detection without Target Domain Adaptation](https://mlsanity.com/paper/2212.07010/) (2022)
- [Cross-Platform Video Person ReID: A New Benchmark Dataset and Adaptation Approach](https://mlsanity.com/paper/2408.07500/) (2024)
- [Person Re-identification: A Retrospective on Domain Specific Open Challenges and Future Trends](https://mlsanity.com/paper/2202.13121/) (2022)
- [GAMA: Generative Adversarial Multi-Object Scene Attacks](https://mlsanity.com/paper/2209.09502/) (2022)
- [Graph Convolution Based Efficient Re-Ranking for Visual Retrieval](https://mlsanity.com/paper/2306.08792/) (2023)
- [Keypoint Message Passing for Video-based Person Re-Identification](https://mlsanity.com/paper/2111.08279/) (2021)
- [Video Person Re-identification using Attribute-enhanced Features](https://mlsanity.com/paper/2108.06946/) (2021)
- [Not Every Patch is Needed: Towards a More Efficient and Effective Backbone for Video-based Person Re-identification](https://mlsanity.com/paper/2501.16811/) (2025)
- [Learning Sequence Descriptor based on Spatio-Temporal Attention for Visual Place Recognition](https://mlsanity.com/paper/2305.11467/) (2023)
- [Leveraging Local Patch Differences in Multi-Object Scenes for Generative Adversarial Attacks](https://mlsanity.com/paper/2209.09883/) (2022)
- [Multi-Stage Spatio-Temporal Aggregation Transformer for Video Person Re-identification](https://mlsanity.com/paper/2301.00531/) (2023)
- [Pose-Aided Video-based Person Re-Identification via Recurrent Graph Convolutional Network](https://mlsanity.com/paper/2209.11582/) (2022)
- [When Person Re-Identification Meets Event Camera: A Benchmark Dataset and An Attribute-guided Re-Identification Framework](https://mlsanity.com/paper/2507.13659/) (2025)
- [Camera Alignment and Weighted Contrastive Learning for Domain Adaptation in Video Person ReID](https://mlsanity.com/paper/2211.03626/) (2022)
- [Deep Learning for Video-based Person Re-Identification: A Survey](https://mlsanity.com/paper/2303.11332/) (2023)
- [Negative Samples are at Large: Leveraging Hard-distance Elastic Loss for Re-identification](https://mlsanity.com/paper/2207.09884/) (2022)
- [VILLS -- Video-Image Learning to Learn Semantics for Person Re-Identification](https://mlsanity.com/paper/2311.17074/) (2023)
- [A High-Accuracy Unsupervised Person Re-identification Method Using Auxiliary Information Mined from Datasets](https://mlsanity.com/paper/2205.03124/) (2022)
- [Context Sensing Attention Network for Video-based Person Re-identification](https://mlsanity.com/paper/2207.02631/) (2022)
- [Robust Person Identification: A WiFi Vision-based Approach](https://mlsanity.com/paper/2210.00127/) (2022)
- [Robust Ensemble Person Re-Identification via Orthogonal Fusion with Occlusion Handling](https://mlsanity.com/paper/2404.00107/) (2024)
- [Beyond Pedestrians: Caption-Guided CLIP Framework for High-Difficulty Video-based Person Re-Identification](https://mlsanity.com/paper/2604.07740/) (2026)
- [Spatial-Temporal Expert Learning for Video-based Person Re-identification](https://mlsanity.com/paper/2607.01353/) (2026)
- [Collaborative Feature Aggregation for Face Super-Resolution and Robust Re-Identification](https://mlsanity.com/paper/2607.28130/) (2026)
- [Paths: Prompt-aware Spatio-temporal Transformer with Hierarchical Multi-modal Fusion for RGB-Event Video Person Re-Identification](https://mlsanity.com/paper/2608.13092/) (2026)

## Abstract (arXiv)

Despite much recent progress in video-based person re-identification (re-ID), the current state-of-the-art still suffers from common real-world challenges such as appearance similarity among various people, occlusions, and frame misalignment. To alleviate these problems, we propose Spatio-Temporal Representation Factorization (STRF), a flexible new computational unit that can be used in conjunction with most existing 3D convolutional neural network architectures for re-ID. The key innovations of STRF over prior work include explicit pathways for learning discriminative temporal and spatial features, with each component further factorized to capture complementary person-specific appearance and motion information. Specifically, temporal factorization comprises two branches, one each for static features (e.g., the color of clothes) that do not change much over time, and dynamic features (e.g., walking patterns) that change over time. Further, spatial factorization also comprises two branches to learn both global (coarse segments) as well as local (finer segments) appearance features, with the local features particularly useful in cases of occlusion or spatial misalignment. These two factorization operations taken together result in a modular architecture for our parameter-wise light STRF unit that can be plugged in between any two 3D convolutional layers, resulting in an end-to-end learning framework. We empirically show that STRF improves performance of various existing baseline architectures while demonstrating new state-of-the-art results using standard person re-ID evaluation protocols on three benchmarks.