# PCIE_Interaction Solution for Ego4D Social Interaction Challenge

arXiv: https://arxiv.org/abs/2505.24404 · HTML: https://mlsanity.com/paper/2505.24404/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 7 references that are in the map.
- First posted on arXiv 30 May 2025.

## Builds on (most cited first, 7 of 7)

- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks](https://mlsanity.com/paper/2312.14238/) (2023)
- [Ego4D: Around the World in 3,000 Hours of Egocentric Video](https://mlsanity.com/paper/2110.07058/) (2021)
- [How far are we from solving the 2D & 3D Face Alignment problem? (and a dataset of 230,000 3D facial landmarks)](https://mlsanity.com/paper/1703.07332/) (2017)
- [NAFSSR: Stereo Image Super-Resolution Using NAFNet](https://mlsanity.com/paper/2204.08714/) (2022)
- [QuAVF: Quality-aware Audio-Visual Fusion for Ego4D Talking to Me Challenge](https://mlsanity.com/paper/2306.17404/) (2023)
- [PCIE_LAM Solution for Ego4D Looking At Me Challenge](https://mlsanity.com/paper/2406.12211/) (2024)

## Built on by (most cited first, 1 of 1)

- [Selective Attention System (SAS): Device-Addressed Speech Detection for Real-Time On-Device Voice AI](https://mlsanity.com/paper/2604.08412/) (2026)

## Abstract (arXiv)

This report presents our team's PCIE_Interaction solution for the Ego4D Social Interaction Challenge at CVPR 2025, addressing both Looking At Me (LAM) and Talking To Me (TTM) tasks. The challenge requires accurate detection of social interactions between subjects and the camera wearer, with LAM relying exclusively on face crop sequences and TTM combining speaker face crops with synchronized audio segments. In the LAM track, we employ face quality enhancement and ensemble methods. For the TTM task, we extend visual interaction analysis by fusing audio and visual cues, weighted by a visual quality score. Our approach achieved 0.81 and 0.71 mean average precision (mAP) on the LAM and TTM challenges leader board. Code is available at https://github.com/KanokphanL/PCIE_Ego4D_Social_Interaction