# Joint Person Segmentation and Identification in Synchronized First- and Third-person Videos

arXiv: https://arxiv.org/abs/1803.11217 · HTML: https://mlsanity.com/paper/1803.11217/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Mar 2018.

## Built on by (most cited first, 18 of 18)

- [Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectives](https://mlsanity.com/paper/2311.18259/) (2023)
- [MeMOT: Multi-Object Tracking with Memory](https://mlsanity.com/paper/2203.16761/) (2022)
- [Ego-Exo: Transferring Visual Representations from Third-person to First-person Videos](https://mlsanity.com/paper/2104.07905/) (2021)
- [EgoExoLearn: A Dataset for Bridging Asynchronous Ego- and Exo-centric View of Procedural Activities in Real World](https://mlsanity.com/paper/2403.16182/) (2024)
- [EgoLife: Towards Egocentric Life Assistant](https://mlsanity.com/paper/2503.03803/) (2025)
- [Retrieval-Augmented Egocentric Video Captioning](https://mlsanity.com/paper/2401.00789/) (2024)
- [Put Myself in Your Shoes: Lifting the Egocentric Perspective from Exocentric Videos](https://mlsanity.com/paper/2403.06351/) (2024)
- [Semi-TCL: Semi-Supervised Track Contrastive Representation Learning](https://mlsanity.com/paper/2107.02396/) (2021)
- [Cross-view Action Recognition Understanding From Exocentric to Egocentric Perspective](https://mlsanity.com/paper/2305.15699/) (2023)
- [LongViTU: Instruction Tuning for Long-Form Video Understanding](https://mlsanity.com/paper/2501.05037/) (2025)
- [Bridging Perspectives: A Survey on Cross-view Collaborative Intelligence with Egocentric-Exocentric Vision](https://mlsanity.com/paper/2506.06253/) (2025)
- [Egocentric and Exocentric Methods: A Short Survey](https://mlsanity.com/paper/2410.20621/) (2024)
- [CAMOT: Camera Angle-aware Multi-Object Tracking](https://mlsanity.com/paper/2409.17533/) (2024)
- [From My View to Yours: Learning Egocentric Cues from Exocentric Video using Privileged Egocentric Supervision](https://mlsanity.com/paper/2501.05711/) (2025)
- [VGGT-Segmentor: Geometry-Enhanced Cross-View Segmentation](https://mlsanity.com/paper/2604.13596/) (2026)
- [DOMR: Establishing Cross-View Segmentation via Dense Object Matching](https://mlsanity.com/paper/2508.04050/) (2025)
- [Towards Imbalanced Motion: Part-Decoupling Network for Video Portrait Segmentation](https://mlsanity.com/paper/2307.16565/) (2023)
- [Sequence-Based Identification of First-Person Camera Wearers in Third-Person Views](https://mlsanity.com/paper/2506.00394/) (2025)

## Abstract (arXiv)

In a world of pervasive cameras, public spaces are often captured from multiple perspectives by cameras of different types, both fixed and mobile. An important problem is to organize these heterogeneous collections of videos by finding connections between them, such as identifying correspondences between the people appearing in the videos and the people holding or wearing the cameras. In this paper, we wish to solve two specific problems: (1) given two or more synchronized third-person videos of a scene, produce a pixel-level segmentation of each visible person and identify corresponding people across different views (i.e., determine who in camera A corresponds with whom in camera B), and (2) given one or more synchronized third-person videos as well as a first-person video taken by a mobile or wearable camera, segment and identify the camera wearer in the third-person videos. Unlike previous work which requires ground truth bounding boxes to estimate the correspondences, we perform person segmentation and identification jointly. We find that solving these two problems simultaneously is mutually beneficial, because better fine-grained segmentation allows us to better perform matching across views, and information from multiple views helps us perform more accurate segmentation. We evaluate our approach on two challenging datasets of interacting people captured from multiple wearable cameras, and show that our proposed method performs significantly better than the state-of-the-art on both person segmentation and identification.