mlsanityData to 7 Oct 2026

Leveraging triplet loss for unsupervised action segmentation

arXiv 2304.06403 ↗First posted 13 Apr 2023cs.CV cs.AIcited by 10

Not on the map. Why?

Abstract (arXiv)

In this paper, we propose a novel fully unsupervised framework that learns action representations suitable for the action segmentation task from the single input video itself, without requiring any training data. Our method is a deep metric learning approach rooted in a shallow network with a triplet loss operating on similarity distributions and a novel triplet selection strategy that effectively models temporal and semantic priors to discover actions in the new representational space. Under these circumstances, we successfully recover temporal boundaries in the learned action representations with higher quality compared with existing unsupervised approaches. The proposed method is evaluated on two widely used benchmark datasets for the action segmentation task and it achieves competitive performance by applying a generic clustering algorithm on the learned representations.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 24 of its 33 references that are in the map. First posted on arXiv 13 Apr 2023.

Builds on

24 of 33 references.

References · 24

  1. 2016Temporal Convolutional Networks for Action Segmentation and Detectioncited by 504
  2. 2019MS-TCN: Multi-Stage Temporal Convolutional Network for Action Segmentationcited by 370
  3. 2015Beyond Short Snippets: Deep Networks for Video Classificationcited by 233
  4. 2015Unsupervised Learning from Narrated Instruction Videoscited by 116
  5. 2019Efficient Parameter-free Clustering Using First Neighbor Relationscited by 96
  6. 2016Segmental Spatiotemporal CNNs for Fine-grained Action Segmentationcited by 78
  7. 2019Unsupervised learning of action classes with continuous temporal embeddingcited by 75
  8. 2018Weakly-Supervised Action Segmentation with Iterative Soft Boundary Assignmentcited by 74
  9. 2016Connectionist Temporal Modeling for Weakly Supervised Action Labelingcited by 70
  10. 2017Weakly Supervised Action Learning with RNN based Fine-to-coarse Modelingcited by 70
  11. 2018NeuralNetwork-Viterbi: A Framework for Weakly Supervised Video Learningcited by 67
  12. 2015An end-to-end generative framework for video segmentation and recognitioncited by 61
12 more
  1. 2018Unsupervised Learning and Segmentation of Complex Activities from Videocited by 61
  2. 2021Temporally-Weighted Hierarchical Clustering for Unsupervised Action Segmentationcited by 58
  3. 2019Weakly Supervised Energy-Based Learning for Action Segmentationcited by 53
  4. 2021Unsupervised Action Segmentation by Joint Representation Learning and Online Clusteringcited by 48
  5. 2016Weakly supervised learning of actions from transcriptscited by 46
  6. 2020Joint Visual-Temporal Embedding for Unsupervised Learning of Actions in Untrimmed Sequencescited by 45
  7. 2018A Perceptual Prediction Framework for Self Supervised Event Segmentationcited by 40
  8. 2020SCT: Set Constrained Temporal Transformer for Set Supervised Action Segmentationcited by 39
  9. 2021Action Shuffle Alternating Learning for Unsupervised Action Segmentationcited by 33
  10. 2021Iterative Contrast-Classify For Semi-supervised Temporal Action Segmentationcited by 19
  11. 2021SSCAP: Self-supervised Co-occurrence Action Parsing for Unsupervised Temporal Action Segmentationcited by 15
  12. 2019Enhancing temporal segmentation by nonlocal self-similaritycited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 3; 2026: 5.

10 papers.

  1. 2025CLOT: Closed Loop Optimal Transport for Unsupervised Action Segmentationcited by 6
  2. 2025Skeleton Motion Words for Unsupervised Skeleton-Based Temporal Action Segmentationcited by 6
  3. 2024Open-Vocabulary Action Localization with Iterative Visual Promptingcited by 5
  4. 20242by2: Weakly-Supervised Learning for Global Action Segmentationcited by 3
  5. 2025Improving action segmentation via explicit similarity measurementcited by 1
  6. 2026Exploring Vision-Language Models for Open-Vocabulary Zero-Shot Action Segmentationcited by 1
  7. 2026Boundary-Centric Clip-Budgeted Active Learning for Temporal Action Segmentationcited by 0
  8. 2026Deep kernel video approximation for unsupervised action segmentationcited by 0
  9. 2026D-CLOT: Double Closed Loop Optimal Transport for Unsupervised Action Segmentationcited by 0
  10. 2026FIS-OT: Feature-Induced Optimal Transport for Unsupervised Action Segmentationcited by 0