mlsanityData to 7 Oct 2026

Timestamp-Supervised Action Segmentation with Graph Convolutional Networks

arXiv 2206.15031 ↗First posted 30 Jun 2022cs.CVcited by 18

Not on the map. Why?

Abstract (arXiv)

We introduce a novel approach for temporal activity segmentation with timestamp supervision. Our main contribution is a graph convolutional network, which is learned in an end-to-end manner to exploit both frame features and connections between neighboring frames to generate dense framewise labels from sparse timestamp labels. The generated dense framewise labels can then be used to train the segmentation model. In addition, we propose a framework for alternating learning of both the segmentation model and the graph convolutional model, which first initializes and then iteratively refines the learned models. Detailed experiments on four public datasets, including 50 Salads, GTEA, Breakfast, and Desktop Assembly, show that our method is superior to the multi-layer perceptron baseline, while performing on par with or better than the state of the art in temporal activity segmentation with timestamp supervision.

Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). Builds on 40 of its 58 references that are in the map. First posted on arXiv 30 Jun 2022.

Builds on

40 of 58 references.

References · 40

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  3. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
  4. 2017Non-local Neural Networkscited by 2,290
  5. 2017A Closer Look at Spatiotemporal Convolutions for Action Recognitioncited by 1,139
  6. 2018Scaling Egocentric Vision: The EPIC-KITCHENS Datasetcited by 789
  7. 2019Memorizing Normality to Detect Anomaly: Memory-augmented Deep Autoencoder for Unsupervised Anomaly Detectioncited by 560
  8. 2019MS-TCN: Multi-Stage Temporal Convolutional Network for Action Segmentationcited by 370
  9. 2014Deeply-Supervised Netscited by 364
  10. 2019Graph Convolutional Networks for Temporal Action Localizationcited by 208
  11. 2018Rethinking the Faster R-CNN Architecture for Temporal Action Localizationcited by 194
  12. 2015Unsupervised Learning from Narrated Instruction Videoscited by 116
28 more
  1. 2020Alleviating Over-segmentation Errors by Detecting Action Boundariescited by 86
  2. 2019D3TW: Discriminative Differentiable Dynamic Time Warping for Weakly Supervised Action Alignment and Segmentationcited by 83
  3. 2019Unsupervised learning of action classes with continuous temporal embeddingcited by 75
  4. 2018Weakly-Supervised Action Segmentation with Iterative Soft Boundary Assignmentcited by 74
  5. 2016Connectionist Temporal Modeling for Weakly Supervised Action Labelingcited by 70
  6. 2017Weakly Supervised Action Learning with RNN based Fine-to-coarse Modelingcited by 70
  7. 2018NeuralNetwork-Viterbi: A Framework for Weakly Supervised Video Learningcited by 67
  8. 2020SF-Net: Single-Frame Supervision for Temporal Action Localizationcited by 62
  9. 2015An end-to-end generative framework for video segmentation and recognitioncited by 61
  10. 2019Weakly Supervised Energy-Based Learning for Action Segmentationcited by 53
  11. 2021Temporal Action Segmentation from Timestamp Supervisioncited by 48
  12. 2021Unsupervised Action Segmentation by Joint Representation Learning and Online Clusteringcited by 48
  13. 2021Learning by Aligning Videos in Timecited by 47
  14. 2016Weakly supervised learning of actions from transcriptscited by 46
  15. 2020Joint Visual-Temporal Embedding for Unsupervised Learning of Actions in Untrimmed Sequencescited by 45
  16. 2020SCT: Set Constrained Temporal Transformer for Set Supervised Action Segmentationcited by 39
  17. 2020Set-Constrained Viterbi for Set-Supervised Action Segmentationcited by 35
  18. 2021Action Shuffle Alternating Learning for Unsupervised Action Segmentationcited by 33
  19. 2015What's Cookin'? Interpreting Cooking Videos using Text, Speech and Visioncited by 31
  20. 2019Action Recognition from Single Timestamp Supervision in Untrimmed Videoscited by 31
  21. 2019Fast Weakly Supervised Action Segmentation Using Mutual Consistencycited by 30
  22. 2015Unsupervised Semantic Parsing of Video Collectionscited by 26
  23. 2016Deep Supervision with Shape Concepts for Occlusion-Aware 3D Object Parsingcited by 19
  24. 2018Deep Supervision with Intermediate Conceptscited by 19
  25. 2020Towards Anomaly Detection in Dashcam Videoscited by 16
  26. 2018Hierarchical Metric Learning and Matching for 2D and 3D Geometric Correspondencescited by 11
  27. 2019Degeneracy in Self-Calibration Revisited and a Deep Learning Solution for Uncalibrated SLAMcited by 10
  28. 2017Healthcare Roboticscited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 5; 2023: 4; 2024: 1; 2025: 2; 2026: 6.

18 papers.

Sort:

By area:

  1. 2022Temporal Action Segmentation: An Analysis of Modern Techniquescited by 85
  2. 2022Inductive and Transductive Few-Shot Video Classification via Appearance and Temporal Alignmentscited by 22
  3. 2023Permutation-Aware Action Segmentation via Unsupervised Frame-to-Segment Alignmentcited by 17
  4. 2025Joint Self-Supervised Video Alignment and Action Segmentationcited by 11
  5. 2023Timestamp-supervised Wearable-based Activity Segmentation and Recognition with Contrastive Learning and Order-Preserving Optimal Transportcited by 10
  6. 2022Semantic2Graph: Graph-based Multi-modal Feature Fusion for Action Segmentation in Videoscited by 9
  7. 2023Action Segmentation Using 2D Skeleton Heatmaps and Multi-Modality Fusioncited by 6
  8. 2022Timestamp-Supervised Action Segmentation from the Perspective of Clusteringcited by 4
  9. 2023Learning by Aligning 2D Skeleton Sequences and Multi-Modality Fusioncited by 3
  10. 2022Bringing Online Egocentric Action Recognition into the wildcited by 1
  11. 2025Looking into the Unknown: Exploring Action Discovery for Segmentation of Known and Unknown Actionscited by 1
  12. 2026Exploring Vision-Language Models for Open-Vocabulary Zero-Shot Action Segmentationcited by 1
6 more
  1. 2026Hierarchical Action Learning for Weakly-Supervised Action Segmentationcited by 1
  2. 2026Target-Guided Selective Reweighting for Physics-Informed Neural Network Inverse Problems: A Transfer Learning Approachcited by 1
  3. 2024Robust Surgical Phase Recognition From Annotation Efficient Supervisioncited by 0
  4. 2026Boundary-Centric Clip-Budgeted Active Learning for Temporal Action Segmentationcited by 0
  5. 2026Unsupervised Skeleton-Based Action Segmentation via Hierarchical Spatiotemporal Vector Quantizationcited by 0
  6. 2026Boundary Voting Network for Ambiguity-Aware Timestamp-Supervised Action Segmentationcited by 0