mlsanityData to 14 Oct 2026

Temporal Convolutional Networks for Action Segmentation and Detection

arXiv 1611.05267 ↗First posted 16 Nov 2016cs.CVcited by 508

Not on the map. Why?

Abstract (arXiv)

The ability to identify and temporally segment fine-grained human actions throughout a video is crucial for robotics, surveillance, education, and beyond. Typical approaches decouple this problem by first extracting local spatiotemporal features from video frames and then feeding them into a temporal classifier that captures high-level temporal patterns. We introduce a new class of temporal models, which we call Temporal Convolutional Networks (TCNs), that use a hierarchy of temporal convolutions to perform fine-grained action segmentation or detection. Our Encoder-Decoder TCN uses pooling and upsampling to efficiently capture long-range temporal patterns whereas our Dilated TCN uses dilated convolutions. We show that TCNs are capable of capturing action compositions, segment durations, and long-range dependencies, and are over a magnitude faster to train than competing LSTM-based Recurrent Neural Networks. We apply these models to three challenging fine-grained datasets and show large improvements over the state of the art.

Key facts. Cited by 508 papers in the map (data through 14 Oct 2026). First posted on arXiv 16 Nov 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 85; 2022: 74; 2023: 91; 2024: 67; 2025: 98; 2026: 93.

508 papers.

Sort:

By area:

  1. 2021NExT-QA:Next Phase of Question-Answering to Explaining Temporal Actionscited by 744
  2. 2021Synthesis of Compositional Animations from Textual Descriptionscited by 178
  3. 2022PromptCast: A New Prompt-based Learning Paradigm for Time Series Forecastingcited by 172
  4. 2021ASFormer: Transformer for Action Segmentationcited by 168
  5. 2021End-to-end Temporal Action Detection with Transformercited by 151
  6. 2023A Survey on Graph Neural Networks for Time Series: Forecasting, Classification, Imputation, and Anomaly Detectioncited by 145
  7. 2021T2VLAD: Global-Local Sequence Alignment for Text-Video Retrievalcited by 103
  8. 2024UniAnimate: Taming Unified Video Diffusion Models for Consistent Human Image Animationcited by 102
  9. 2021Trans-SVNet: Accurate Phase Recognition from Surgical Videos via Hybrid Embedding Aggregation Transformercited by 94
  10. 2022Temporal Action Segmentation: An Analysis of Modern Techniquescited by 85
  11. 2022Temporal Alignment Networks for Long-term Videocited by 84
  12. 2023Time Series as Images: Vision Transformer for Irregularly Sampled Time Seriescited by 79
38 more
  1. 2023Diffusion Action Segmentationcited by 74
  2. 2021Object-Region Video Transformerscited by 73
  3. 2021Improving Deep Learning Interpretability by Saliency Guided Trainingcited by 68
  4. 2021Temporal Memory Relation Network for Workflow Recognition from Surgical Videocited by 66
  5. 2021Temporally-Weighted Hierarchical Clustering for Unsupervised Action Segmentationcited by 58
  6. 2022Future Transformer for Long-term Action Anticipationcited by 58
  7. 2022Unified Fully and Timestamp Supervised Temporal Action Segmentation via Sequence to Sequence Translationcited by 55
  8. 2022Bridge-Prompt: Towards Ordinal Action Understanding in Instructional Videoscited by 50
  9. 2021Towards Unified Surgical Skill Assessmentcited by 49
  10. 2021Activity Graph Transformer for Temporal Action Localizationcited by 48
  11. 2021Temporal Action Segmentation from Timestamp Supervisioncited by 48
  12. 2024Frequency Adaptive Normalization For Non-stationary Time Series Forecastingcited by 47
  13. 2021A Survey of Sound Source Localization with Deep Learning Methodscited by 46
  14. 2022Spatio-Temporal Graph Few-Shot Learning with Cross-City Knowledge Transfercited by 43
  15. 2021Global2Local: Efficient Structure Search for Video Action Segmentationcited by 42
  16. 2021MS-TCT: Multi-Scale Temporal ConvTransformer for Action Detectioncited by 42
  17. 2022An Extensive Data Processing Pipeline for MIMIC-IVcited by 40
  18. 2021MM-Pyramid: Multimodal Pyramid Attentional Network for Audio-Visual Event Localization and Video Parsingcited by 39
  19. 2023VidChapters-7M: Video Chapters at Scalecited by 39
  20. 2024Temporally Consistent Unbalanced Optimal Transport for Unsupervised Action Segmentationcited by 38
  21. 2021Predicting the Future from First Person (Egocentric) Vision: A Surveycited by 36
  22. 2021DCAN: Improving Temporal Action Detection via Dual Context Aggregationcited by 36
  23. 2023How Much Temporal Long-Term Context is Needed for Action Segmentation?cited by 36
  24. 2024Time Series Modeling for Heart Rate Prediction: From ARIMA to Transformerscited by 35
  25. 2023Visual Tuningcited by 34
  26. 2023Optical flow estimation from event-based cameras and spiking neural networkscited by 33
  27. 2021Learning an Augmented RGB Representation with Cross-Modal Knowledge Distillation for Action Detectioncited by 32
  28. 2021Deep Learning-based Action Detection in Untrimmed Videos: A Surveycited by 32
  29. 2021Self-Regulated Learning for Egocentric Video Activity Anticipationcited by 32
  30. 2021The Multi-Agent Behavior Dataset: Mouse Dyadic Social Interactionscited by 31
  31. 2022Action Quality Assessment with Temporal Parsing Transformercited by 31
  32. 2022Uncertainty Quantification of Sparse Travel Demand Prediction with Spatial-Temporal Graph Neural Networkscited by 29
  33. 2021Coarse-Fine Networks for Temporal Activity Detection in Videoscited by 28
  34. 2021Coarse to Fine Multi-Resolution Temporal Convolutional Networkcited by 28
  35. 2022A Survey on Video Action Recognition in Sports: Datasets, Methods and Applicationscited by 28
  36. 2023Model scale versus domain knowledge in statistical forecasting of chaotic systemscited by 27
  37. 2021Unsupervised Discriminative Embedding for Sub-Action Learning in Complex Activitiescited by 26
  38. 2021Foreground-Action Consistency Network for Weakly Supervised Temporal Action Localizationcited by 26

50 most cited of 508 All 508 (JSON)