# Learning a Weakly-Supervised Video Actor-Action Segmentation Model with a Wise Selection

arXiv: https://arxiv.org/abs/2003.13141 · HTML: https://mlsanity.com/paper/2003.13141/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Mar 2020.

## Built on by (most cited first, 3 of 3)

- [Pedestrian Attribute Recognition in Video Surveillance Scenarios Based on View-attribute Attention Localization](https://mlsanity.com/paper/2106.06485/) (2021)
- [U-LanD: Uncertainty-Driven Video Landmark Detection](https://mlsanity.com/paper/2102.01586/) (2021)
- [Action-Slot: Structured Action-Centric Representation Learning for Multi-Agent Atomic Activity Understanding](https://mlsanity.com/paper/2609.24127/) (2026)

## Abstract (arXiv)

We address weakly-supervised video actor-action segmentation (VAAS), which extends general video object segmentation (VOS) to additionally consider action labels of the actors. The most successful methods on VOS synthesize a pool of pseudo-annotations (PAs) and then refine them iteratively. However, they face challenges as to how to select from a massive amount of PAs high-quality ones, how to set an appropriate stop condition for weakly-supervised training, and how to initialize PAs pertaining to VAAS. To overcome these challenges, we propose a general Weakly-Supervised framework with a Wise Selection of training samples and model evaluation criterion (WS^2). Instead of blindly trusting quality-inconsistent PAs, WS^2 employs a learning-based selection to select effective PAs and a novel region integrity criterion as a stopping condition for weakly-supervised training. In addition, a 3D-Conv GCAM is devised to adapt to the VAAS task. Extensive experiments show that WS^2 achieves state-of-the-art performance on both weakly-supervised VOS and VAAS tasks and is on par with the best fully-supervised method on VAAS.