mlsanityData to 7 Oct 2026

MS-TCN: Multi-Stage Temporal Convolutional Network for Action Segmentation

arXiv 1903.01945 ↗First posted 5 Mar 2019cs.CVcited by 370

Not on the map. Why?

Abstract (arXiv)

Temporally locating and classifying action segments in long untrimmed videos is of particular interest to many applications like surveillance and robotics. While traditional approaches follow a two-step pipeline, by generating frame-wise probabilities and then feeding them to high-level temporal models, recent approaches use temporal convolutions to directly classify the video frames. In this paper, we introduce a multi-stage architecture for the temporal action segmentation task. Each stage features a set of dilated temporal convolutions to generate an initial prediction that is refined by the next one. This architecture is trained using a combination of a classification loss and a proposed smoothing loss that penalizes over-segmentation errors. Extensive evaluation shows the effectiveness of the proposed model in capturing long-range dependencies and recognizing action segments. Our model achieves state-of-the-art results on three challenging datasets: 50Salads, Georgia Tech Egocentric Activities (GTEA), and the Breakfast dataset.

Key facts. Cited by 370 papers in the map (data through 7 Oct 2026). First posted on arXiv 5 Mar 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 49; 2022: 65; 2023: 65; 2024: 60; 2025: 71; 2026: 60.

370 papers.

Sort:

By area:

  1. 2021Multi-Stage Progressive Image Restorationcited by 813
  2. 2022HOI4D: A 4D Egocentric Dataset for Category-Level Human-Object Interactioncited by 321
  3. 2022MAXIM: Multi-Axis MLP for Image Processingcited by 301
  4. 2022Assembly101: A Large-Scale Multi-View Video Dataset for Understanding Procedural Activitiescited by 284
  5. 2021ASFormer: Transformer for Action Segmentationcited by 166
  6. 2023VideoLLM: Modeling Video Sequence with Large Language Modelscited by 116
  7. 2024Video Mamba Suite: State Space Model as a Versatile Alternative for Video Understandingcited by 106
  8. 2021Trans-SVNet: Accurate Phase Recognition from Surgical Videos via Hybrid Embedding Aggregation Transformercited by 93
  9. 2024EgoExoLearn: A Dataset for Bridging Asynchronous Ego- and Exo-centric View of Procedural Activities in Real Worldcited by 93
  10. 2022Temporal Action Segmentation: An Analysis of Modern Techniquescited by 85
  11. 2022Temporal Alignment Networks for Long-term Videocited by 84
  12. 2023Diffusion Action Segmentationcited by 74
38 more
  1. 2023Learning Multi-modal Representations by Watching Hundreds of Surgical Video Lecturescited by 68
  2. 2022Zero-Shot Temporal Action Detection via Vision-Language Promptingcited by 65
  3. 2021Generic Event Boundary Detection: A Benchmark for Event Segmentationcited by 64
  4. 2025Scalable Vision-Language-Action Model Pretraining for Robotic Manipulation with Real-Life Human Activity Videoscited by 63
  5. 2022Future Transformer for Long-term Action Anticipationcited by 57
  6. 2023LoViT: Long Video Transformer for Surgical Phase Recognitioncited by 56
  7. 2022Unified Fully and Timestamp Supervised Temporal Action Segmentation via Sequence to Sequence Translationcited by 55
  8. 2023SAR-RARP50: Segmentation of surgical instrumentation and Action Recognition on Robot-Assisted Radical Prostatectomy Challengecited by 54
  9. 2022Bridge-Prompt: Towards Ordinal Action Understanding in Instructional Videoscited by 50
  10. 2021Towards Unified Surgical Skill Assessmentcited by 49
  11. 2021Temporal Action Segmentation from Timestamp Supervisioncited by 48
  12. 2021Unsupervised Action Segmentation by Joint Representation Learning and Online Clusteringcited by 48
  13. 2021Learning by Aligning Videos in Timecited by 47
  14. 2023CaptainCook4D: A Dataset for Understanding Errors in Procedural Activitiescited by 47
  15. 2022CholecTriplet2021: A benchmark challenge for surgical action triplet recognitioncited by 44
  16. 2021Global2Local: Efficient Structure Search for Video Action Segmentationcited by 42
  17. 2021Multi-Task Temporal Convolutional Networks for Joint Recognition of Surgical Phases and Steps in Gastric Bypass Procedurescited by 42
  18. 2022Spotting Temporally Precise, Fine-Grained Events in Videocited by 40
  19. 2024MambaMixer: Efficient Selective State Space Models with Dual Token and Channel Selectioncited by 40
  20. 2021MM-Pyramid: Multimodal Pyramid Attentional Network for Audio-Visual Event Localization and Video Parsingcited by 39
  21. 2023VidChapters-7M: Video Chapters at Scalecited by 39
  22. 2024Temporally Consistent Unbalanced Optimal Transport for Unsupervised Action Segmentationcited by 37
  23. 2021Predicting the Future from First Person (Egocentric) Vision: A Surveycited by 36
  24. 2023How Much Temporal Long-Term Context is Needed for Action Segmentation?cited by 36
  25. 2021Learning an Augmented RGB Representation with Cross-Modal Knowledge Distillation for Action Detectioncited by 32
  26. 2021Vision Transformers with Hierarchical Attentioncited by 31
  27. 2022Proposal-Free Temporal Action Detection via Global Segmentation Mask Learningcited by 29
  28. 2021Coarse to Fine Multi-Resolution Temporal Convolutional Networkcited by 28
  29. 2023TemporalMaxer: Maximize Temporal Context with only Max Pooling for Temporal Action Localizationcited by 27
  30. 2021Unsupervised Discriminative Embedding for Sub-Action Learning in Complex Activitiescited by 26
  31. 2021Joint Visual Semantic Reasoning: Multi-Stage Decoder for Text Recognitioncited by 24
  32. 2021Exploring Segment-level Semantics for Online Phase Recognition from Surgical Videoscited by 23
  33. 2022C2F-TCN: A Framework for Semi and Fully Supervised Temporal Action Segmentationcited by 23
  34. 2022Self-supervised Transformer for Deepfake Detectioncited by 22
  35. 2021Action Segmentation with Mixed Temporal Domain Adaptationcited by 21
  36. 2021Long Short View Feature Decomposition via Contrastive Video Representation Learningcited by 21
  37. 2021Background-Click Supervision for Temporal Action Localizationcited by 21
  38. 2021Not End-to-End: Explore Multi-Stage Architecture for Online Surgical Phase Recognitioncited by 20

50 most cited of 370 All 370 (JSON)