mlsanityData to 7 Oct 2026

Leveraging Action Affinity and Continuity for Semi-supervised Temporal Action Segmentation

arXiv 2207.08653 ↗First posted 18 Jul 2022cs.CV cs.AIcited by 17

Not on the map. Why?

Abstract (arXiv)

We present a semi-supervised learning approach to the temporal action segmentation task. The goal of the task is to temporally detect and segment actions in long, untrimmed procedural videos, where only a small set of videos are densely labelled, and a large collection of videos are unlabelled. To this end, we propose two novel loss functions for the unlabelled data: an action affinity loss and an action continuity loss. The action affinity loss guides the unlabelled samples learning by imposing the action priors induced from the labelled set. Action continuity loss enforces the temporal continuity of actions, which also provides frame-wise classification supervision. In addition, we propose an Adaptive Boundary Smoothing (ABS) approach to build coarser action boundaries for more robust and reliable learning. The proposed loss functions and ABS were evaluated on three benchmarks. Results show that they significantly improved action segmentation performance with a low amount (5% and 10%) of labelled data and achieved comparable results to full supervision with 50% labelled data. Furthermore, ABS succeeded in boosting performance when integrated into fully-supervised learning.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). Builds on 25 of its 36 references that are in the map. First posted on arXiv 18 Jul 2022.

Builds on

25 of 36 references.

References · 25

  1. 2016Temporal Ensembling for Semi-Supervised Learningcited by 1,054
  2. 2019Pseudo-Labeling and Confirmation Bias in Deep Semi-Supervised Learningcited by 507
  3. 2016Temporal Convolutional Networks for Action Segmentation and Detectioncited by 504
  4. 2017Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning resultscited by 470
  5. 2019MS-TCN: Multi-Stage Temporal Convolutional Network for Action Segmentationcited by 370
  6. 2021In Defense of Pseudo-Labeling: An Uncertainty-Aware Pseudo-label Selection Framework for Semi-Supervised Learningcited by 301
  7. 2019Label Propagation for Deep Semi-supervised Learningcited by 287
  8. 2020Alleviating Over-segmentation Errors by Detecting Action Boundariescited by 86
  9. 2019D3TW: Discriminative Differentiable Dynamic Time Warping for Weakly Supervised Action Alignment and Segmentationcited by 83
  10. 2021Re-distributing Biased Pseudo Labels for Semi-supervised Semantic Segmentation: A Baseline Investigationcited by 78
  11. 2019Unsupervised learning of action classes with continuous temporal embeddingcited by 75
  12. 2018Weakly-Supervised Action Segmentation with Iterative Soft Boundary Assignmentcited by 74
13 more
  1. 2020Action Segmentation with Joint Self-Supervised Temporal Domain Adaptationcited by 74
  2. 2018Unsupervised Learning and Segmentation of Complex Activities from Videocited by 61
  3. 2021Temporally-Weighted Hierarchical Clustering for Unsupervised Action Segmentationcited by 58
  4. 2021Temporal Action Segmentation from Timestamp Supervisioncited by 48
  5. 2021Data-Uncertainty Guided Multi-Phase Learning for Semi-Supervised Object Detectioncited by 48
  6. 2016Weakly supervised learning of actions from transcriptscited by 46
  7. 2020SCT: Set Constrained Temporal Transformer for Set Supervised Action Segmentationcited by 39
  8. 2021Self-Supervised Learning for Semi-Supervised Temporal Action Proposalcited by 39
  9. 2021Action Shuffle Alternating Learning for Unsupervised Action Segmentationcited by 33
  10. 2021Coarse to Fine Multi-Resolution Temporal Convolutional Networkcited by 28
  11. 2021Iterative Contrast-Classify For Semi-supervised Temporal Action Segmentationcited by 19
  12. 2021Pixel-Level Bijective Matching for Video Object Segmentationcited by 11
  13. 2018Feature Affinity based Pseudo Labeling for Semi-supervised Person Re-identificationcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 1; 2024: 4; 2025: 6; 2026: 5.

17 papers.

Sort:

By area:

  1. 2022Temporal Action Segmentation: An Analysis of Modern Techniquescited by 85
  2. 2024Efficient and Effective Weakly-Supervised Action Segmentation via Action-Transition-Aware Boundary Alignmentcited by 13
  3. 2024OnlineTAS: An Online Baseline for Temporal Action Segmentationcited by 7
  4. 2024Coherent Temporal Synthesis for Incremental Action Segmentationcited by 5
  5. 2025Condensing Action Segmentation Datasets via Generative Network Inversioncited by 4
  6. 2024Long-Tail Temporal Action Segmentation with Group-wise Temporal Logit Adjustmentcited by 3
  7. 2023SMC-NCA: Semantic-guided Multi-level Contrast for Semi-supervised Temporal Action Segmentationcited by 1
  8. 2025Improving action segmentation via explicit similarity measurementcited by 1
  9. 2025A Temporal Modeling Framework for Video Pre-Training on Video Instance Segmentationcited by 1
  10. 2025Looking into the Unknown: Exploring Action Discovery for Segmentation of Known and Unknown Actionscited by 1
  11. 2026Exploring Vision-Language Models for Open-Vocabulary Zero-Shot Action Segmentationcited by 1
  12. 2025Data Augmentation for Instruction Following Policies via Trajectory Segmentationcited by 0
5 more
  1. 2025Unsupervised Discovery of Long-Term Spatiotemporal Periodic Workflows in Human Activitiescited by 0
  2. 2026Boundary-Centric Clip-Budgeted Active Learning for Temporal Action Segmentationcited by 0
  3. 2026Jointly Learning Structured Representations and Stabilized Affinity for Human Motion Segmentationcited by 0
  4. 2026Improving Temporal Action Segmentation via Constraint-Aware Decodingcited by 0
  5. 2026Adaptive Latent Trajectory Anchoring for Action Segmentation Dataset Condensationcited by 0