# Learning from Synchronization: Self-Supervised Uncalibrated Multi-View Person Association in Challenging Scenes

arXiv: https://arxiv.org/abs/2503.13739 · HTML: https://mlsanity.com/paper/2503.13739/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 42 references that are in the map.
- First posted on arXiv 17 Mar 2025.

## Builds on (most cited first, 20 of 20)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [MonoPair: Monocular 3D Object Detection Using Pairwise Spatial Relationships](https://mlsanity.com/paper/2003.00504/) (2020)
- [Fast and Robust Multi-Person 3D Pose Estimation from Multiple Views](https://mlsanity.com/paper/1901.04111/) (2019)
- [Learning Generalisable Omni-Scale Representations for Person Re-Identification](https://mlsanity.com/paper/1910.06827/) (2019)
- [MVOR: A Multi-view RGB-D Operating Room Dataset for 2D and 3D Human Pose Estimation](https://mlsanity.com/paper/1808.08180/) (2018)
- [4D Association Graph for Realtime Multi-person Motion Capture Using Multiple Video Cameras](https://mlsanity.com/paper/2002.12625/) (2020)
- [Cross-View Tracking for Multi-Human 3D Pose Estimation at over 100 FPS](https://mlsanity.com/paper/2003.03972/) (2020)
- [CycAs: Self-supervised Cycle Association for Learning Re-identifiable Descriptions](https://mlsanity.com/paper/2007.07577/) (2020)
- [Progressive End-to-End Object Detection in Crowded Scenes](https://mlsanity.com/paper/2203.07669/) (2022)
- [Shape-aware Multi-Person Pose Estimation from Multi-View Images](https://mlsanity.com/paper/2110.02330/) (2021)
- [ReST: A Reconfigurable Spatial-Temporal Graph Model for Multi-Camera Multi-Object Tracking](https://mlsanity.com/paper/2308.13229/) (2023)
- [Multi-person 3D Pose Estimation in Crowded Scenes Based on Multi-View Geometry](https://mlsanity.com/paper/2007.10986/) (2020)
- [Ego2Top: Matching Viewers in Egocentric and Top-view Videos](https://mlsanity.com/paper/1607.06986/) (2016)
- [MessyTable: Instance Association in Multiple Camera Views](https://mlsanity.com/paper/2007.14878/) (2020)
- [Graph Neural Networks for Cross-Camera Data Association](https://mlsanity.com/paper/2201.06311/) (2022)
- [Self-supervised Multi-view Person Association and Its Applications](https://mlsanity.com/paper/1805.08717/) (2018)
- [Self-Supervised Multi-View Synchronization Learning for 3D Pose Estimation](https://mlsanity.com/paper/2010.06218/) (2020)
- [A Unified Multi-view Multi-person Tracking Framework](https://mlsanity.com/paper/2302.03820/) (2023)
- [Multi-View Person Matching and 3D Pose Estimation with Arbitrary Uncalibrated Camera Networks](https://mlsanity.com/paper/2312.01561/) (2023)

## Built on by (most cited first, 7 of 7)

- [LocalScore: Local Density-Aware Similarity Scoring for Biometrics](https://mlsanity.com/paper/2602.01012/) (2026)
- [COMPOSE: Hypergraph Cover Optimization for Multi-view 3D Human Pose Estimation](https://mlsanity.com/paper/2601.09698/) (2026)
- [SapiensID 2.0: Aligning Human Recognition Foundation Models with Human Perception](https://mlsanity.com/paper/2608.10497/) (2026)
- [Ranking vs. Assignment: The Metric Mismatch in Multi-View Object Association](https://mlsanity.com/paper/2606.02022/) (2026)
- [AI for Quality Assurance in the Operating Room](https://mlsanity.com/paper/2606.30657/) (2026)
- [Beyond Appearance: A Multi-cue Framework and Large-scale Benchmark for Pedestrian Association and Tracking on Mobile Aerial-Ground Platforms](https://mlsanity.com/paper/2607.23803/) (2026)
- [TrackFish3D: Self-Supervised 3D Tracking of Schooling Fish from Multi-view Videos](https://mlsanity.com/paper/2609.38347/) (2026)

## Abstract (arXiv)

Multi-view person association is a fundamental step towards multi-view analysis of human activities. Although the person re-identification features have been proven effective, they become unreliable in challenging scenes where persons share similar appearances. Therefore, cross-view geometric constraints are required for a more robust association. However, most existing approaches are either fully-supervised using ground-truth identity labels or require calibrated camera parameters that are hard to obtain. In this work, we investigate the potential of learning from synchronization, and propose a self-supervised uncalibrated multi-view person association approach, Self-MVA, without using any annotations. Specifically, we propose a self-supervised learning framework, consisting of an encoder-decoder model and a self-supervised pretext task, cross-view image synchronization, which aims to distinguish whether two images from different views are captured at the same time. The model encodes each person's unified geometric and appearance features, and we train it by utilizing synchronization labels for supervision after applying Hungarian matching to bridge the gap between instance-wise and image-wise distances. To further reduce the solution space, we propose two types of self-supervised linear constraints: multi-view re-projection and pairwise edge association. Extensive experiments on three challenging public benchmark datasets (WILDTRACK, MVOR, and SOLDIERS) show that our approach achieves state-of-the-art results, surpassing existing unsupervised and fully-supervised approaches. Code is available at https://github.com/CAMMA-public/Self-MVA.