# HopaDIFF: Holistic-Partial Aware Fourier Conditioned Diffusion for Referring Human Action Segmentation in Multi-Person Scenarios

arXiv: https://arxiv.org/abs/2506.09650 · HTML: https://mlsanity.com/paper/2506.09650/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Jun 2025.

## Built on by (most cited first, 5 of 5)

- [IMPACT-CYCLE: A Contract-Based Multi-Agent System for Claim-Level Supervisory Correction of Long-Video Semantic Memory](https://mlsanity.com/paper/2604.20136/) (2026)
- [IMPACT-HOI: Supervisory Control for Onset-Anchored Partial HOI Event Construction](https://mlsanity.com/paper/2605.01666/) (2026)
- [Learning Human Visual Attention on 3D Surfaces through Geometry-Queried Semantic Priors](https://mlsanity.com/paper/2602.06419/) (2026)
- [InterEdit: Navigating Text-Guided 3D Dyadic Human Motion Editing](https://mlsanity.com/paper/2603.13082/) (2026)
- [IMPACT-Scribe: Interactive Temporal Action Segmentation with Boundary Scribbles and Query Planning](https://mlsanity.com/paper/2605.01668/) (2026)

## Abstract (arXiv)

Action segmentation is a core challenge in high-level video understanding, aiming to partition untrimmed videos into segments and assign each a label from a predefined action set. Existing methods primarily address single-person activities with fixed action sequences, overlooking multi-person scenarios. In this work, we pioneer textual reference-guided human action segmentation in multi-person settings, where a textual description specifies the target person for segmentation. We introduce the first dataset for Referring Human Action Segmentation, i.e., RHAS133, built from 133 movies and annotated with 137 fine-grained actions with 33h video data, together with textual descriptions for this new task. Benchmarking existing action segmentation methods on RHAS133 using VLM-based feature extractors reveals limited performance and poor aggregation of visual cues for the target person. To address this, we propose a holistic-partial aware Fourier-conditioned diffusion framework, i.e., HopaDIFF, leveraging a novel cross-input gate attentional xLSTM to enhance holistic-partial long-range reasoning and a novel Fourier condition to introduce more fine-grained control to improve the action segmentation generation. HopaDIFF achieves state-of-the-art results on RHAS133 in diverse evaluation settings. The dataset and code are available at https://github.com/KPeng9510/HopaDIFF.