mlsanityData to 7 Oct 2026

Unsupervised Learning and Segmentation of Complex Activities from Video

arXiv 1803.09490 ↗First posted 26 Mar 2018cs.CVcited by 61

Not on the map. Why?

Abstract (arXiv)

This paper presents a new method for unsupervised segmentation of complex activities from video into multiple steps, or sub-activities, without any textual input. We propose an iterative discriminative-generative approach which alternates between discriminatively learning the appearance of sub-activities from the videos' visual features to sub-activity labels and generatively modelling the temporal structure of sub-activities using a Generalized Mallows Model. In addition, we introduce a model for background to account for frames unrelated to the actual activities. Our approach is validated on the challenging Breakfast Actions and Inria Instructional Videos datasets and outperforms both unsupervised and weakly-supervised state of the art.

Key facts. Cited by 61 papers in the map (data through 7 Oct 2026). First posted on arXiv 26 Mar 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 20; 2022: 15; 2023: 6; 2024: 7; 2025: 7; 2026: 6.

61 papers.

Sort:

By area:

  1. 2022Assembly101: A Large-Scale Multi-View Video Dataset for Understanding Procedural Activitiescited by 284
  2. 2022Temporal Action Segmentation: An Analysis of Modern Techniquescited by 85
  3. 2021Temporally-Weighted Hierarchical Clustering for Unsupervised Action Segmentationcited by 58
  4. 2021Unsupervised Action Segmentation by Joint Representation Learning and Online Clusteringcited by 48
  5. 2021Global2Local: Efficient Structure Search for Video Action Segmentationcited by 42
  6. 2022Look for the Change: Learning Object States and State-Modifying Actions from Untrimmed Web Videoscited by 39
  7. 2022TL;DW? Summarizing Instructional Videos with Task Relevance & Cross-Modal Saliencycited by 35
  8. 2021Action Shuffle Alternating Learning for Unsupervised Action Segmentationcited by 33
  9. 2021Deep Learning-based Action Detection in Untrimmed Videos: A Surveycited by 32
  10. 2022Min-Max Similarity: A Contrastive Semi-Supervised Deep Learning Network for Surgical Tools Segmentationcited by 28
  11. 2021Unsupervised Discriminative Embedding for Sub-Action Learning in Complex Activitiescited by 26
  12. 2022C2F-TCN: A Framework for Semi and Fully Supervised Temporal Action Segmentationcited by 23
38 more
  1. 2021Iterative Contrast-Classify For Semi-supervised Temporal Action Segmentationcited by 19
  2. 2024Hierarchical Vector Quantization for Unsupervised Action Segmentationcited by 18
  3. 2022Leveraging Action Affinity and Continuity for Semi-supervised Temporal Action Segmentationcited by 17
  4. 2023Permutation-Aware Action Segmentation via Unsupervised Frame-to-Segment Alignmentcited by 17
  5. 2021Temporal Action Segmentation with High-level Complex Activity Labelscited by 16
  6. 2021Object Priors for Classifying and Localizing Unseen Actionscited by 15
  7. 2021SSCAP: Self-supervised Co-occurrence Action Parsing for Unsupervised Temporal Action Segmentationcited by 15
  8. 2023STEPs: Self-Supervised Key Step Extraction and Localization from Unlabeled Procedural Videoscited by 13
  9. 2022Weakly-Supervised Online Action Segmentation in Multi-View Instructional Videoscited by 12
  10. 2021Hierarchical Modeling for Task Recognition and Action Segmentation in Weakly-Labeled Instructional Videoscited by 11
  11. 2023OTAS: Unsupervised Boundary Detection for Object-Centric Temporal Action Segmentationcited by 11
  12. 2021Transferring Knowledge from Text to Video: Zero-Shot Anticipation for Procedural Actionscited by 10
  13. 2022RF-Next: Efficient Receptive Field Search for Convolutional Neural Networkscited by 10
  14. 2023Leveraging triplet loss for unsupervised action segmentationcited by 10
  15. 2024Learning by Watching: A Review of Video-based Learning Approaches for Robot Manipulationcited by 10
  16. 2021Anchor-Constrained Viterbi for Set-Supervised Action Segmentationcited by 8
  17. 2026ProcVLM: Learning Procedure-Grounded Progress Rewards for Robotic Manipulationcited by 8
  18. 2021Uncertain Process Data with Probabilistic Knowledge: Problem Characterization and Challengescited by 7
  19. 2021CoSeg: Cognitively Inspired Unsupervised Generic Event Segmentationcited by 7
  20. 2022Conformance Checking Over Stochastically Known Logscited by 7
  21. 2024OnlineTAS: An Online Baseline for Temporal Action Segmentationcited by 7
  22. 2025CLOT: Closed Loop Optimal Transport for Unsupervised Action Segmentationcited by 6
  23. 2025Skeleton Motion Words for Unsupervised Skeleton-Based Temporal Action Segmentationcited by 6
  24. 2021Weakly-guided Self-supervised Pretraining for Temporal Activity Detectioncited by 5
  25. 2024Coherent Temporal Synthesis for Incremental Action Segmentationcited by 5
  26. 2021Unsupervised Action Segmentation for Instructional Videoscited by 4
  27. 2022Transformers in Action: Weakly Supervised Action Segmentationcited by 4
  28. 2025Condensing Action Segmentation Datasets via Generative Network Inversioncited by 4
  29. 2021Towards Tokenized Human Dynamics Representationcited by 3
  30. 20242by2: Weakly-Supervised Learning for Global Action Segmentationcited by 3
  31. 2021Unsupervised Discovery of Actions in Instructional Videoscited by 2
  32. 2022Text-Driven Video Acceleration: A Weakly-Supervised Reinforcement Learning Methodcited by 2
  33. 2022SVGraph: Learning Semantic Graphs from Instructional Videoscited by 2
  34. 2023TAEC: Unsupervised Action Segmentation with Temporal-Aware Embedding and Clusteringcited by 2
  35. 2022A Generalized & Robust Framework For Timestamp Supervision in Temporal Action Segmentationcited by 1
  36. 2022Learning Video-independent Eye Contact Segmentation from In-the-Wild Videoscited by 1
  37. 2024Temporal and Semantic Evaluation Metrics for Foundation Models in Post-Hoc Analysis of Robotic Sub-taskscited by 1
  38. 2025Improving action segmentation via explicit similarity measurementcited by 1

50 most cited of 61 All 61 (JSON)