mlsanityData to 14 Oct 2026

2by2: Weakly-Supervised Learning for Global Action Segmentation

arXiv 2412.12829 ↗First posted 17 Dec 2024cs.CVcited by 3

Not on the map. Why?

Abstract (arXiv)

This paper presents a simple yet effective approach for the poorly investigated task of global action segmentation, aiming at grouping frames capturing the same action across videos of different activities. Unlike the case of videos depicting all the same activity, the temporal order of actions is not roughly shared among all videos, making the task even more challenging. We propose to use activity labels to learn, in a weakly-supervised fashion, action representations suitable for global action segmentation. For this purpose, we introduce a triadic learning approach for video pairs, to ensure intra-video action discrimination, as well as inter-video and inter-activity action association. For the backbone architecture, we use a Siamese network based on sparse transformers that takes as input video pairs and determine whether they belong to the same activity. The proposed approach is validated on two challenging benchmark datasets: Breakfast and YouTube Instructions, outperforming state-of-the-art methods.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 36 of its 50 references that are in the map. First posted on arXiv 17 Dec 2024.

Builds on

36 of 50 references.

References · 36

  1. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  2. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,270
  3. 2017Time-Contrastive Networks: Self-Supervised Learning from Videocited by 459
  4. 2017Look, Listen and Learncited by 458
  5. 2019MS-TCN: Multi-Stage Temporal Convolutional Network for Action Segmentationcited by 373
  6. 2017Soft-DTW: a Differentiable Loss Function for Time-Seriescited by 271
  7. 2021ASFormer: Transformer for Action Segmentationcited by 168
  8. 2019Temporal Cycle-Consistency Learningcited by 154
  9. 2015Unsupervised Learning from Narrated Instruction Videoscited by 116
  10. 2019Efficient Parameter-free Clustering Using First Neighbor Relationscited by 97
  11. 2022Temporal Action Segmentation: An Analysis of Modern Techniquescited by 85
  12. 2019Unsupervised learning of action classes with continuous temporal embeddingcited by 76
24 more
  1. 2018Unsupervised Learning and Segmentation of Complex Activities from Videocited by 61
  2. 2021Temporally-Weighted Hierarchical Clustering for Unsupervised Action Segmentationcited by 58
  3. 2022Unified Fully and Timestamp Supervised Temporal Action Segmentation via Sequence to Sequence Translationcited by 55
  4. 2021Unsupervised Action Segmentation by Joint Representation Learning and Online Clusteringcited by 49
  5. 2021Learning by Aligning Videos in Timecited by 48
  6. 2020Joint Visual-Temporal Embedding for Unsupervised Learning of Actions in Untrimmed Sequencescited by 45
  7. 2015Recognizing Fine-Grained and Composite Activities using Hand-Centric Features and Script Datacited by 40
  8. 2018A Perceptual Prediction Framework for Self Supervised Event Segmentationcited by 40
  9. 2021Drop-DTW: Aligning Common Signal Between Sequences While Dropping Outlierscited by 39
  10. 2024Temporally Consistent Unbalanced Optimal Transport for Unsupervised Action Segmentationcited by 38
  11. 2023How Much Temporal Long-Term Context is Needed for Action Segmentation?cited by 36
  12. 2021Action Shuffle Alternating Learning for Unsupervised Action Segmentationcited by 33
  13. 2019Fast Weakly Supervised Action Segmentation Using Mutual Consistencycited by 30
  14. 2021Unsupervised Discriminative Embedding for Sub-Action Learning in Complex Activitiescited by 26
  15. 2021Learning Discriminative Prototypes with Dynamic Time Warpingcited by 23
  16. 2021Learning to Align Sequential Actions in the Wildcited by 20
  17. 2023Permutation-Aware Action Segmentation via Unsupervised Frame-to-Segment Alignmentcited by 17
  18. 2021Temporal Action Segmentation with High-level Complex Activity Labelscited by 16
  19. 2021SSCAP: Self-supervised Co-occurrence Action Parsing for Unsupervised Temporal Action Segmentationcited by 15
  20. 2024Efficient and Effective Weakly-Supervised Action Segmentation via Action-Transition-Aware Boundary Alignmentcited by 13
  21. 2023OTAS: Unsupervised Boundary Detection for Object-Centric Temporal Action Segmentationcited by 11
  22. 2023Leveraging triplet loss for unsupervised action segmentationcited by 10
  23. 2024ViSTec: Video Modeling for Sports Technique Recognition and Tactical Analysiscited by 7
  24. 2024Depth Over RGB: Automatic Evaluation of Open Surgery Skills Using Depth Cameracited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2025CLOT: Closed Loop Optimal Transport for Unsupervised Action Segmentationcited by 6
  2. 2026Hierarchical Action Learning for Weakly-Supervised Action Segmentationcited by 1
  3. 2026D-CLOT: Double Closed Loop Optimal Transport for Unsupervised Action Segmentationcited by 0