# A Novel end-to-end Framework for Occluded Pixel Reconstruction with Spatio-temporal Features for Improved Person Re-identification

arXiv: https://arxiv.org/abs/2304.07721 · HTML: https://mlsanity.com/paper/2304.07721/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 21 references that are in the map.
- First posted on arXiv 16 Apr 2023.

## Builds on (most cited first, 9 of 9)

- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting](https://mlsanity.com/paper/1506.04214/) (2015)
- [High-Order Information Matters: Learning Relation and Topology for Occluded Person Re-Identification](https://mlsanity.com/paper/2003.08177/) (2020)
- [Learning Generalisable Omni-Scale Representations for Person Re-Identification](https://mlsanity.com/paper/1910.06827/) (2019)
- [Pose-guided Visible Part Matching for Occluded Person ReID](https://mlsanity.com/paper/2004.00230/) (2020)
- [Deep Spatial Feature Reconstruction for Partial Person Re-identification: Alignment-Free Approach](https://mlsanity.com/paper/1801.00881/) (2018)
- [Foreground-aware Pyramid Reconstruction for Alignment-free Occluded Person Re-identification](https://mlsanity.com/paper/1904.04975/) (2019)
- [STA: Spatial-Temporal Attention for Large-Scale Video-based Person Re-Identification](https://mlsanity.com/paper/1811.04129/) (2018)
- [STNReID : Deep Convolutional Networks with Pairwise Spatial Transformer Networks for Partial Person Re-identification](https://mlsanity.com/paper/1903.07072/) (2019)

## Built on by (most cited first, 2 of 2)

- [Deep Learning-based Spatio Temporal Facial Feature Visual Speech Recognition](https://mlsanity.com/paper/2305.00552/) (2023)
- [CoVid-19 Detection leveraging Vision Transformers and Explainable AI](https://mlsanity.com/paper/2307.16033/) (2023)

## Abstract (arXiv)

Person re-identification is vital for monitoring and tracking crowd movement to enhance public security. However, re-identification in the presence of occlusion substantially reduces the performance of existing systems and is a challenging area. In this work, we propose a plausible solution to this problem by developing effective occlusion detection and reconstruction framework for RGB images/videos consisting of Deep Neural Networks. Specifically, a CNN-based occlusion detection model classifies individual input frames, followed by a Conv-LSTM and Autoencoder to reconstruct the occluded pixels corresponding to the occluded frames for sequential (video) and non-sequential (image) data, respectively. The quality of the reconstructed RGB frames is further refined and fine-tuned using a Conditional Generative Adversarial Network (cGAN). Our method is evaluated on four well-known public data sets of the domain, and the qualitative reconstruction results are indeed appealing. Quantitative evaluation in terms of re-identification accuracy of the Siamese network showed an exceptional Rank-1 accuracy after occluded pixel reconstruction on various datasets. A comparative analysis with state-of-the-art approaches also demonstrates the robustness of our work for use in real-life surveillance systems.