# Coupled Generative Adversarial Network for Continuous Fine-grained Action Segmentation

arXiv: https://arxiv.org/abs/1909.09283 · HTML: https://mlsanity.com/paper/1909.09283/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Sep 2019.

## Built on by (most cited first, 7 of 7)

- [Temporal Action Segmentation: An Analysis of Modern Techniques](https://mlsanity.com/paper/2210.10352/) (2022)
- [Diffusion Action Segmentation](https://mlsanity.com/paper/2303.17959/) (2023)
- [Global2Local: Efficient Structure Search for Video Action Segmentation](https://mlsanity.com/paper/2101.00910/) (2021)
- [Action Segmentation with Mixed Temporal Domain Adaptation](https://mlsanity.com/paper/2104.07461/) (2021)
- [Continuous Human Action Recognition for Human-Machine Interaction: A Review](https://mlsanity.com/paper/2202.13096/) (2022)
- [RF-Next: Efficient Receptive Field Search for Convolutional Neural Networks](https://mlsanity.com/paper/2206.06637/) (2022)
- [Dual-Stream Alignment for Action Segmentation](https://mlsanity.com/paper/2510.07652/) (2025)

## Abstract (arXiv)

We propose a novel conditional GAN (cGAN) model for continuous fine-grained human action segmentation, that utilises multi-modal data and learned scene context information. The proposed approach utilises two GANs: termed Action GAN and Auxiliary GAN, where the Action GAN is trained to operate over the current RGB frame while the Auxiliary GAN utilises supplementary information such as depth or optical flow. The goal of both GANs is to generate similar `action codes', a vector representation of the current action. To facilitate this process a context extractor that incorporates data and recent outputs from both modes is used to extract context information to aid recognition. The result is a recurrent GAN architecture which learns a task specific loss function from multiple feature modalities. Extensive evaluations on variants of the proposed model to show the importance of utilising different information streams such as context and auxiliary information in the proposed network; and show that our model is capable of outperforming state-of-the-art methods for three widely used datasets: 50 Salads, MERL Shopping and Georgia Tech Egocentric Activities, comprising both static and dynamic camera settings.