# VILLS -- Video-Image Learning to Learn Semantics for Person Re-Identification

arXiv: https://arxiv.org/abs/2311.17074 · HTML: https://mlsanity.com/paper/2311.17074/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 43 of its 69 references that are in the map.
- First posted on arXiv 27 Nov 2023.

## Builds on (most cited first, 30 of 43)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Flamingo: a Visual Language Model for Few-Shot Learning](https://mlsanity.com/paper/2204.14198/) (2022)
- [Improved Baselines with Momentum Contrastive Learning](https://mlsanity.com/paper/2003.04297/) (2020)
- [Perceiver: General Perception with Iterative Attention](https://mlsanity.com/paper/2103.03206/) (2021)
- [A Survey on Contrastive Self-supervised Learning](https://mlsanity.com/paper/2011.00362/) (2020)
- [MViTv2: Improved Multiscale Vision Transformers for Classification and Detection](https://mlsanity.com/paper/2112.01526/) (2021)
- [TransReID: Transformer-based Object Re-Identification](https://mlsanity.com/paper/2102.04378/) (2021)
- [A Short Note on the Kinetics-700 Human Action Dataset](https://mlsanity.com/paper/1907.06987/) (2019)
- [Self-Supervised Representation Learning: Introduction, Advances and Challenges](https://mlsanity.com/paper/2110.09327/) (2021)
- [Relation-Aware Global Attention for Person Re-identification](https://mlsanity.com/paper/1904.02998/) (2019)
- [Gait Recognition via Effective Global-Local Feature Representation and Local Temporal Aggregation](https://mlsanity.com/paper/2011.01461/) (2020)
- [Spreading vectors for similarity search](https://mlsanity.com/paper/1806.03198/) (2018)
- [Identity-Guided Human Semantic Parsing for Person Re-Identification](https://mlsanity.com/paper/2007.13467/) (2020)
- [Clothes-Changing Person Re-identification with RGB Modality Only](https://mlsanity.com/paper/2204.06890/) (2022)
- [Beyond Appearance: a Semantic Controllable Self-Supervised Learning Framework for Human-Centric Visual Tasks](https://mlsanity.com/paper/2303.17602/) (2023)
- [Person Re-identification by Contour Sketch under Moderate Clothing Change](https://mlsanity.com/paper/2002.02295/) (2020)
- [Long-Term Cloth-Changing Person Re-identification](https://mlsanity.com/paper/2005.12633/) (2020)
- [CLIP-Driven Fine-grained Text-Image Person Re-identification](https://mlsanity.com/paper/2210.10276/) (2022)
- [Identifying Unknown Instances for Autonomous Driving](https://mlsanity.com/paper/1910.11296/) (2019)
- [Cloth-Changing Person Re-identification from A Single Image with Gait Prediction and Regularization](https://mlsanity.com/paper/2103.15537/) (2021)
- [Pose-guided Feature Disentangling for Occluded Person Re-identification Based on Transformer](https://mlsanity.com/paper/2112.02466/) (2021)
- [Expanding Accurate Person Recognition to New Altitudes and Ranges: The BRIAR Dataset](https://mlsanity.com/paper/2211.01917/) (2022)
- [UniHCP: A Unified Model for Human-Centric Perceptions](https://mlsanity.com/paper/2303.02936/) (2023)
- [Instruct-ReID: A Multi-purpose Person Re-identification Task with Instructions](https://mlsanity.com/paper/2306.07520/) (2023)
- [Watching You: Global-guided Reciprocal Learning for Video-based Person Re-identification](https://mlsanity.com/paper/2103.04337/) (2021)

## Built on by (most cited first, 2 of 2)

- [Not All Starting Points Are Equal: Pre-trained Priors and Their Outsized Impact on Person Identification](https://mlsanity.com/paper/2507.17640/) (2025)
- [SapiensID 2.0: Aligning Human Recognition Foundation Models with Human Perception](https://mlsanity.com/paper/2608.10497/) (2026)

## Abstract (arXiv)

Person Re-identification is a research area with significant real world applications. Despite recent progress, existing methods face challenges in robust re-identification in the wild, e.g., by focusing only on a particular modality and on unreliable patterns such as clothing. A generalized method is highly desired, but remains elusive to achieve due to issues such as the trade-off between spatial and temporal resolution and imperfect feature extraction. We propose VILLS (Video-Image Learning to Learn Semantics), a self-supervised method that jointly learns spatial and temporal features from images and videos. VILLS first designs a local semantic extraction module that adaptively extracts semantically consistent and robust spatial features. Then, VILLS designs a unified feature learning and adaptation module to represent image and video modalities in a consistent feature space. By Leveraging self-supervised, large-scale pre-training, VILLS establishes a new State-of-The-Art that significantly outperforms existing image and video-based methods.