# Fine-grained Action Segmentation using the Semi-Supervised Action GAN

arXiv: https://arxiv.org/abs/1909.09269 · HTML: https://mlsanity.com/paper/1909.09269/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Sep 2019.

## Built on by (most cited first, 12 of 12)

- [ASFormer: Transformer for Action Segmentation](https://mlsanity.com/paper/2110.08568/) (2021)
- [Unified Fully and Timestamp Supervised Temporal Action Segmentation via Sequence to Sequence Translation](https://mlsanity.com/paper/2209.00638/) (2022)
- [How Much Temporal Long-Term Context is Needed for Action Segmentation?](https://mlsanity.com/paper/2308.11358/) (2023)
- [Continuous Human Action Recognition for Human-Machine Interaction: A Review](https://mlsanity.com/paper/2202.13096/) (2022)
- [Deep Learning Approaches for Seizure Video Analysis: A Review](https://mlsanity.com/paper/2312.10930/) (2023)
- [Temporal Segment Transformer for Action Segmentation](https://mlsanity.com/paper/2302.13074/) (2023)
- [Hand Hygiene Assessment via Joint Step Segmentation and Key Action Scorer](https://mlsanity.com/paper/2209.12221/) (2022)
- [DIR-AS: Decoupling Individual Identification and Temporal Reasoning for Action Segmentation](https://mlsanity.com/paper/2304.02110/) (2023)
- [Improving action segmentation via explicit similarity measurement](https://mlsanity.com/paper/2502.10713/) (2025)
- [Dual-Stream Alignment for Action Segmentation](https://mlsanity.com/paper/2510.07652/) (2025)
- [Exploring Vision-Language Models for Open-Vocabulary Zero-Shot Action Segmentation](https://mlsanity.com/paper/2602.21406/) (2026)
- [Conditional Generative Data-free Knowledge Distillation](https://mlsanity.com/paper/2112.15358/) (2021)

## Abstract (arXiv)

In this paper we address the problem of continuous fine-grained action segmentation, in which multiple actions are present in an unsegmented video stream. The challenge for this task lies in the need to represent the hierarchical nature of the actions and to detect the transitions between actions, allowing us to localise the actions within the video effectively. We propose a novel recurrent semi-supervised Generative Adversarial Network (GAN) model for continuous fine-grained human action segmentation. Temporal context information is captured via a novel Gated Context Extractor (GCE) module, composed of gated attention units, that directs the queued context information through the generator model, for enhanced action segmentation. The GAN is made to learn features in a semi-supervised manner, enabling the model to perform action classification jointly with the standard, unsupervised, GAN learning procedure. We perform extensive evaluations on different architectural variants to demonstrate the importance of the proposed network architecture, and show that it is capable of outperforming current state-of-the-art on three challenging datasets: 50 Salads, MERL Shopping and Georgia Tech Egocentric Activities dataset.