# Video-based Person Re-identification with Spatial and Temporal Memory Networks

arXiv: https://arxiv.org/abs/2108.09039 · HTML: https://mlsanity.com/paper/2108.09039/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 49 references that are in the map.
- First posted on arXiv 20 Aug 2021.

## Builds on (most cited first, 30 of 34)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [Random Erasing Data Augmentation](https://mlsanity.com/paper/1708.04896/) (2017)
- [In Defense of the Triplet Loss for Person Re-Identification](https://mlsanity.com/paper/1703.07737/) (2017)
- [Performance Measures and a Data Set for Multi-Target, Multi-Camera Tracking](https://mlsanity.com/paper/1609.01775/) (2016)
- [Memorizing Normality to Detect Anomaly: Memory-augmented Deep Autoencoder for Unsupervised Anomaly Detection](https://mlsanity.com/paper/1904.02639/) (2019)
- [Video Object Segmentation using Space-Time Memory Networks](https://mlsanity.com/paper/1904.00607/) (2019)
- [Deeply-Supervised Nets](https://mlsanity.com/paper/1409.5185/) (2014)
- [Memory Networks](https://mlsanity.com/paper/1410.3916/) (2014)
- [Learning Memory-guided Normality for Anomaly Detection](https://mlsanity.com/paper/2003.13228/) (2020)
- [Key-Value Memory Networks for Directly Reading Documents](https://mlsanity.com/paper/1606.03126/) (2016)
- [Joint Discriminative and Generative Learning for Person Re-identification](https://mlsanity.com/paper/1904.07223/) (2019)
- [Harmonious Attention Network for Person Re-Identification](https://mlsanity.com/paper/1802.08122/) (2018)
- [Invariance Matters: Exemplar Memory for Domain Adaptive Person Re-identification](https://mlsanity.com/paper/1904.01990/) (2019)
- [Part-Aligned Bilinear Representations for Person Re-identification](https://mlsanity.com/paper/1804.07094/) (2018)
- [HydraPlus-Net: Attentive Deep Features for Pedestrian Analysis](https://mlsanity.com/paper/1709.09930/) (2017)
- [Pose-driven Deep Convolutional Model for Person Re-identification](https://mlsanity.com/paper/1709.08325/) (2017)
- [Memory Matching Networks for One-Shot Image Recognition](https://mlsanity.com/paper/1804.08281/) (2018)
- [Learning Multi-Granular Hypergraphs for Video-Based Person Re-Identification](https://mlsanity.com/paper/2104.14913/) (2021)
- [Diversity Regularized Spatiotemporal Attention for Video-based Person Re-identification](https://mlsanity.com/paper/1803.09882/) (2018)
- [Temporal Complementary Learning for Video Person Re-Identification](https://mlsanity.com/paper/2007.09357/) (2020)
- [STA: Spatial-Temporal Attention for Large-Scale Video-based Person Re-Identification](https://mlsanity.com/paper/1811.04129/) (2018)
- [Multi-Granularity Reference-Aided Attentive Feature Aggregation for Video-based Person Re-identification](https://mlsanity.com/paper/2003.12224/) (2020)
- [Multi-scale 3D Convolution Network for Video Based Person Re-Identification](https://mlsanity.com/paper/1811.07468/) (2018)
- [Learning Disentangled Representation for Robust Person Re-identification](https://mlsanity.com/paper/1910.12003/) (2019)
- [Training Deeper Convolutional Networks with Deep Supervision](https://mlsanity.com/paper/1505.02496/) (2015)
- [Quality Aware Network for Set to Set Recognition](https://mlsanity.com/paper/1704.03373/) (2017)

## Built on by (most cited first, 30 of 35)

- [TF-CLIP: Learning Text-free CLIP for Video-based Person Re-Identification](https://mlsanity.com/paper/2312.09627/) (2023)
- [Learning Modal-Invariant and Temporal-Memory for Video-based Visible-Infrared Person Re-Identification](https://mlsanity.com/paper/2208.02450/) (2022)
- [Deeply-Coupled Convolution-Transformer with Spatial-temporal Complementary Learning for Video-based Person Re-identification](https://mlsanity.com/paper/2304.14122/) (2023)
- [Cross-Platform Video Person ReID: A New Benchmark Dataset and Adaptation Approach](https://mlsanity.com/paper/2408.07500/) (2024)
- [MEVID: Multi-view Extended Videos with Identities for Video Person Re-Identification](https://mlsanity.com/paper/2211.04656/) (2022)
- [Person Re-identification: A Retrospective on Domain Specific Open Challenges and Future Trends](https://mlsanity.com/paper/2202.13121/) (2022)
- [Video-based Person Re-identification with Long Short-Term Representation Learning](https://mlsanity.com/paper/2308.03703/) (2023)
- [ShARc: Shape and Appearance Recognition for Person Identification In-the-wild](https://mlsanity.com/paper/2310.15946/) (2023)
- [AG-VPReID: A Challenging Large-Scale Benchmark for Aerial-Ground Video-based Person Re-Identification](https://mlsanity.com/paper/2503.08121/) (2025)
- [GEFF: Improving Any Clothes-Changing Person ReID Model using Gallery Enrichment with Face Features](https://mlsanity.com/paper/2211.13807/) (2022)
- [HAMoBE: Hierarchical and Adaptive Mixture of Biometric Experts for Video-based Person ReID](https://mlsanity.com/paper/2508.05038/) (2025)
- [Whole-body Detection, Recognition and Identification at Altitude and Range](https://mlsanity.com/paper/2311.05725/) (2023)
- [Activity-Biometrics: Person Identification from Daily Activities](https://mlsanity.com/paper/2403.17360/) (2024)
- [Not Every Patch is Needed: Towards a More Efficient and Effective Backbone for Video-based Person Re-identification](https://mlsanity.com/paper/2501.16811/) (2025)
- [Feature Disentanglement Learning with Switching and Aggregation for Video-based Person Re-Identification](https://mlsanity.com/paper/2212.09498/) (2022)
- [A Benchmark of Video-Based Clothes-Changing Person Re-Identification](https://mlsanity.com/paper/2211.11165/) (2022)
- [When Person Re-Identification Meets Event Camera: A Benchmark Dataset and An Attribute-guided Re-Identification Framework](https://mlsanity.com/paper/2507.13659/) (2025)
- [GorillaWatch: An Automated System for In-the-Wild Gorilla Re-Identification and Population Monitoring](https://mlsanity.com/paper/2512.07776/) (2025)
- [Deep Learning for Video-based Person Re-Identification: A Survey](https://mlsanity.com/paper/2303.11332/) (2023)
- [Adversarial Self-Attack Defense and Spatial-Temporal Relation Mining for Visible-Infrared Video Person Re-Identification](https://mlsanity.com/paper/2307.03903/) (2023)
- [DisenQ: Disentangling Q-Former for Activity-Biometrics](https://mlsanity.com/paper/2507.07262/) (2025)
- [VILLS -- Video-Image Learning to Learn Semantics for Person Re-Identification](https://mlsanity.com/paper/2311.17074/) (2023)
- [Context Sensing Attention Network for Video-based Person Re-identification](https://mlsanity.com/paper/2207.02631/) (2022)
- [Multi Camera Connected Vision System with Multi View Analytics: A Comprehensive Survey](https://mlsanity.com/paper/2510.09731/) (2025)
- [Seeing Across Time and Views: Multi-Temporal Cross-View Learning for Robust Video Person Re-Identification](https://mlsanity.com/paper/2511.02564/) (2025)
- [Cross-Modal Synergies: Unveiling the Potential of Motion-Aware Fusion Networks in Handling Dynamic and Static ReID Scenarios](https://mlsanity.com/paper/2502.00665/) (2025)
- [Causality and "In-the-Wild" Video-Based Person Re-ID: A Survey](https://mlsanity.com/paper/2505.20540/) (2025)
- [S3CE-Net: Spike-guided Spatiotemporal Semantic Coupling and Expansion Network for Long Sequence Event Re-Identification](https://mlsanity.com/paper/2505.24401/) (2025)
- [Online Human Action Detection during Escorting](https://mlsanity.com/paper/2506.23573/) (2025)
- [AG-VPReID.VIR: Bridging Aerial and Ground Platforms for Video-based Visible-Infrared Person Re-ID](https://mlsanity.com/paper/2507.17995/) (2025)

## Abstract (arXiv)

Video-based person re-identification (reID) aims to retrieve person videos with the same identity as a query person across multiple cameras. Spatial and temporal distractors in person videos, such as background clutter and partial occlusions over frames, respectively, make this task much more challenging than image-based person reID. We observe that spatial distractors appear consistently in a particular location, and temporal distractors show several patterns, e.g., partial occlusions occur in the first few frames, where such patterns provide informative cues for predicting which frames to focus on (i.e., temporal attentions). Based on this, we introduce a novel Spatial and Temporal Memory Networks (STMN). The spatial memory stores features for spatial distractors that frequently emerge across video frames, while the temporal memory saves attentions which are optimized for typical temporal patterns in person videos. We leverage the spatial and temporal memories to refine frame-level person representations and to aggregate the refined frame-level features into a sequence-level person representation, respectively, effectively handling spatial and temporal distractors in person videos. We also introduce a memory spread loss preventing our model from addressing particular items only in the memories. Experimental results on standard benchmarks, including MARS, DukeMTMC-VideoReID, and LS-VID, demonstrate the effectiveness of our method.