mlsanityData to 14 Oct 2026

Temporal Convolutional Networks: A Unified Approach to Action Segmentation

arXiv 1608.08242 ↗First posted 29 Aug 2016cs.CVcited by 199

Not on the map. Why?

Abstract (arXiv)

The dominant paradigm for video-based action segmentation is composed of two steps: first, for each frame, compute low-level features using Dense Trajectories or a Convolutional Neural Network that encode spatiotemporal information locally, and second, input these features into a classifier that captures high-level temporal relationships, such as a Recurrent Neural Network (RNN). While often effective, this decoupling requires specifying two separate models, each with their own complexities, and prevents capturing more nuanced long-range spatiotemporal relationships. We propose a unified approach, as demonstrated by our Temporal Convolutional Network (TCN), that hierarchically captures relationships at low-, intermediate-, and high-level time-scales. Our model achieves superior or competitive performance using video or sensor data on three public action segmentation datasets and can be trained in a fraction of the time it takes to train an RNN.

Key facts. Cited by 199 papers in the map (data through 14 Oct 2026). First posted on arXiv 29 Aug 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 23; 2022: 37; 2023: 36; 2024: 32; 2025: 46; 2026: 25.

199 papers.

Sort:

By area:

  1. 2023A decoder-only foundation model for time-series forecastingcited by 692
  2. 2023Large Language Models Are Zero-Shot Time Series Forecasterscited by 500
  3. 2023EmoTalk: Speech-Driven Emotional Disentanglement for 3D Face Animationcited by 142
  4. 2021Rendezvous: Attention Mechanisms for the Recognition of Surgical Action Triplets in Endoscopic Videoscited by 141
  5. 2022Liquid Structural State-Space Modelscited by 136
  6. 2023Large Models for Time Series and Spatio-Temporal Data: A Survey and Outlookcited by 110
  7. 2021Trans-SVNet: Accurate Phase Recognition from Surgical Videos via Hybrid Embedding Aggregation Transformercited by 94
  8. 2021Voice2Series: Reprogramming Acoustic Models for Time Series Classificationcited by 91
  9. 2023Generative Time Series Forecasting with Diffusion, Denoise, and Disentanglementcited by 90
  10. 2021Chaos as an interpretable benchmark for forecasting and data-driven modellingcited by 88
  11. 2023SelfTalk: A Self-Supervised Commutative Training Diagram to Comprehend 3D Talking Facescited by 68
  12. 2023LoViT: Long Video Transformer for Surgical Phase Recognitioncited by 56
38 more
  1. 2023SAR-RARP50: Segmentation of surgical instrumentation and Action Recognition on Robot-Assisted Radical Prostatectomy Challengecited by 56
  2. 2022ADATIME: A Benchmarking Suite for Domain Adaptation on Time Series Datacited by 45
  3. 2021Deep Reinforcement Learning in Computer Vision: A Comprehensive Surveycited by 43
  4. 2021Multi-Task Temporal Convolutional Networks for Joint Recognition of Surgical Phases and Steps in Gastric Bypass Procedurescited by 42
  5. 2025TS-RAG: Retrieval-Augmented Generation based Time Series Foundation Models are Stronger Zero-Shot Forecastercited by 40
  6. 2023Visual Tuningcited by 34
  7. 2023Optical flow estimation from event-based cameras and spiking neural networkscited by 33
  8. 2024A Survey of Deep Learning and Foundation Models for Time Series Forecastingcited by 33
  9. 2024Bootstrapping Reinforcement Learning with Imitation for Vision-Based Agile Flightcited by 31
  10. 2021Unsupervised Speech Enhancement using Dynamical Variational Auto-Encoderscited by 28
  11. 2022Machine Learning for Microcontroller-Class Hardware: A Reviewcited by 28
  12. 2023Learning to Ground Instructional Articles in Videos through Narrationscited by 28
  13. 2023Model scale versus domain knowledge in statistical forecasting of chaotic systemscited by 27
  14. 2023LoCoNet: Long-Short Context Network for Active Speaker Detectioncited by 25
  15. 2022Video-Text Representation Learning via Differentiable Weak Temporal Alignmentcited by 24
  16. 2024Specialized Foundation Models Struggle to Beat Supervised Baselinescited by 24
  17. 2021Gesture Recognition in Robotic Surgery: a Reviewcited by 23
  18. 2026Hy-Embodied-0.5-VLA: From Vision-Language-Action Models to a Real-World Robot Learning Stackcited by 22
  19. 2023Leveraging TCN and Transformer for effective visual-audio fusion in continuous emotion recognitioncited by 20
  20. 2024xLSTM-Mixer: Multivariate Time Series Forecasting by Mixing via Scalar Memoriescited by 19
  21. 2022Physics-Inspired Temporal Learning of Quadrotor Dynamics for Accurate Model Predictive Trajectory Trackingcited by 18
  22. 2023LightCTS: A Lightweight Framework for Correlated Time Series Forecastingcited by 18
  23. 2023Traffic Prediction using Artificial Intelligence: Review of Recent Advances and Emerging Opportunitiescited by 16
  24. 2023Contrastive Learning for Enhancing Robust Scene Transfer in Vision-based Agile Flightcited by 15
  25. 2022Q-PPG: Energy-Efficient PPG-based Heart Rate Monitoring on Wearable Devicescited by 14
  26. 2023Transformer Encoder with Multiscale Deep Learning for Pain Classification Using Physiological Signalscited by 14
  27. 2022Classification of Long Sequential Data using Circular Dilated Convolutional Neural Networkscited by 13
  28. 2024Hybrid2 Neural ODE Causal Modeling and an Application to Glycemic Responsecited by 13
  29. 2022Recognition and Prediction of Surgical Gestures and Trajectories Using Transformer Models in Robot-Assisted Surgerycited by 12
  30. 2024TIGER: Time-frequency Interleaved Gain Extraction and Reconstruction for Efficient Speech Separationcited by 11
  31. 2021Motion Guided Attention Fusion to Recognize Interactions from Videoscited by 10
  32. 2023Weakly Supervised Multi-Task Representation Learning for Human Activity Analysis Using Wearablescited by 10
  33. 2024Cherry-Picking in Time Series Forecasting: How to Select Datasets to Make Your Model Shinecited by 10
  34. 2026The Forecast After the Forecast: A Post-Processing Shift in Time Seriescited by 10
  35. 2021Deep Autoregressive Models with Spectral Attentioncited by 9
  36. 2022ChordMixer: A Scalable Neural Attention Model for Sequences with Different Lengthscited by 9
  37. 2023Heterogeneous Anomaly Detection for Software Systems via Semi-supervised Cross-modal Attentioncited by 9
  38. 2025Towards Identifiability of Hierarchical Temporal Causal Representation Learningcited by 9

50 most cited of 199 All 199 (JSON)