mlsanityData to 7 Oct 2026

Iterative Contrast-Classify For Semi-supervised Temporal Action Segmentation

arXiv 2112.01402 ↗First posted 2 Dec 2021cs.CVcited by 19

Not on the map. Why?

Abstract (arXiv)

Temporal action segmentation classifies the action of each frame in (long) video sequences. Due to the high cost of frame-wise labeling, we propose the first semi-supervised method for temporal action segmentation. Our method hinges on unsupervised representation learning, which, for temporal action segmentation, poses unique challenges. Actions in untrimmed videos vary in length and have unknown labels and start/end times. Ordering of actions across videos may also vary. We propose a novel way to learn frame-wise representations from temporal convolutional networks (TCNs) by clustering input features with added time-proximity condition and multi-resolution similarity. By merging representation learning with conventional supervised learning, we develop an "Iterative-Contrast-Classify (ICC)" semi-supervised learning scheme. With more labelled data, ICC progressively improves in performance; ICC semi-supervised learning, with 40% labelled videos, performs similar to fully-supervised counterparts. Our ICC improves MoF by {+1.8, +5.6, +2.5}% on Breakfast, 50Salads and GTEA respectively for 100% labelled videos.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Builds on 38 of its 47 references that are in the map. First posted on arXiv 2 Dec 2021.

Builds on

38 of 47 references.

References · 38

  1. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  2. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  3. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
  4. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
  5. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,841
  6. 2020Supervised Contrastive Learningcited by 2,803
  7. 2019End-to-End Learning of Visual Representations from Uncurated Instructional Videoscited by 506
  8. 2016Temporal Convolutional Networks for Action Segmentation and Detectioncited by 504
  9. 2017Time-Contrastive Networks: Self-Supervised Learning from Videocited by 453
  10. 2020Spatiotemporal Contrastive Video Representation Learningcited by 372
  11. 2019MS-TCN: Multi-Stage Temporal Convolutional Network for Action Segmentationcited by 370
  12. 2015What's the Point: Semantic Segmentation with Point Supervisioncited by 297
26 more
  1. 2019Self-Supervised Learning by Cross-Modal Audio-Video Clusteringcited by 264
  2. 2021A Large-Scale Study on Unsupervised Spatiotemporal Representation Learningcited by 214
  3. 2018Adversarial Learning for Semi-Supervised Semantic Segmentationcited by 173
  4. 2019Semi-Supervised Semantic Segmentation with High- and Low-level Consistencycited by 147
  5. 2019Efficient Parameter-free Clustering Using First Neighbor Relationscited by 96
  6. 2019D3TW: Discriminative Differentiable Dynamic Time Warping for Weakly Supervised Action Alignment and Segmentationcited by 83
  7. 2019Unsupervised learning of action classes with continuous temporal embeddingcited by 75
  8. 2018Weakly-Supervised Action Segmentation with Iterative Soft Boundary Assignmentcited by 74
  9. 2020Action Segmentation with Joint Self-Supervised Temporal Domain Adaptationcited by 74
  10. 2018NeuralNetwork-Viterbi: A Framework for Weakly Supervised Video Learningcited by 67
  11. 2020SF-Net: Single-Frame Supervision for Temporal Action Localizationcited by 62
  12. 2018Unsupervised Learning and Segmentation of Complex Activities from Videocited by 61
  13. 2021Temporally-Weighted Hierarchical Clustering for Unsupervised Action Segmentationcited by 58
  14. 2019Weakly Supervised Energy-Based Learning for Action Segmentationcited by 53
  15. 2021Semi-Supervised Action Recognition with Temporal Contrastive Learningcited by 50
  16. 2021Temporal Action Segmentation from Timestamp Supervisioncited by 48
  17. 2020Joint Visual-Temporal Embedding for Unsupervised Learning of Actions in Untrimmed Sequencescited by 45
  18. 2020Momentum Contrastive Learning for Few-Shot COVID-19 Diagnosis from Chest CT Imagescited by 43
  19. 2019A Hybrid RNN-HMM Approach for Weakly Supervised Temporal Action Segmentationcited by 36
  20. 2020Cycle-Contrast for Self-Supervised Video Representation Learningcited by 34
  21. 2021Action Shuffle Alternating Learning for Unsupervised Action Segmentationcited by 33
  22. 2019Fast Weakly Supervised Action Segmentation Using Mutual Consistencycited by 30
  23. 2021Coarse to Fine Multi-Resolution Temporal Convolutional Networkcited by 28
  24. 2020Can Temporal Information Help with Contrastive Self-Supervised Learning?cited by 27
  25. 2020Contrastive Transformation for Self-supervised Correspondence Learningcited by 21
  26. 2021Pretrained equivariant features improve unsupervised landmark discoverycited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 3; 2024: 4; 2025: 3; 2026: 6.

19 papers.

Sort:

By area:

  1. 2022Temporal Action Segmentation: An Analysis of Modern Techniquescited by 85
  2. 2022C2F-TCN: A Framework for Semi and Fully Supervised Temporal Action Segmentationcited by 23
  3. 2022Leveraging Action Affinity and Continuity for Semi-supervised Temporal Action Segmentationcited by 17
  4. 2023Synchronization is All You Need: Exocentric-to-Egocentric Transfer for Temporal Action Segmentation with Unlabeled Synchronized Video Pairscited by 17
  5. 2023Leveraging triplet loss for unsupervised action segmentationcited by 10
  6. 2024OnlineTAS: An Online Baseline for Temporal Action Segmentationcited by 7
  7. 2024Coherent Temporal Synthesis for Incremental Action Segmentationcited by 5
  8. 2025Condensing Action Segmentation Datasets via Generative Network Inversioncited by 4
  9. 2026ENIGMA-360: An Ego-Exo Dataset for Human Behavior Understanding in Industrial Scenarioscited by 2
  10. 2023SMC-NCA: Semantic-guided Multi-level Contrast for Semi-supervised Temporal Action Segmentationcited by 1
  11. 2025Improving action segmentation via explicit similarity measurementcited by 1
  12. 2025Looking into the Unknown: Exploring Action Discovery for Segmentation of Known and Unknown Actionscited by 1
7 more
  1. 2024Computer User Interface Understanding. A New Dataset and a Learning Frameworkcited by 0
  2. 2024Robust Surgical Phase Recognition From Annotation Efficient Supervisioncited by 0
  3. 2026Boundary-Centric Clip-Budgeted Active Learning for Temporal Action Segmentationcited by 0
  4. 2026Towards Human Motion World Models via Executable Behaviour Representationscited by 0
  5. 2026Jointly Learning Structured Representations and Stabilized Affinity for Human Motion Segmentationcited by 0
  6. 2026Improving Temporal Action Segmentation via Constraint-Aware Decodingcited by 0
  7. 2026Adaptive Latent Trajectory Anchoring for Action Segmentation Dataset Condensationcited by 0