mlsanityData to 14 Oct 2026

Temporal Action Detection with Structured Segment Networks

arXiv 1704.06228 ↗First posted 20 Apr 2017cs.CVcited by 267

Not on the map. Why?

Abstract (arXiv)

Detecting actions in untrimmed videos is an important yet challenging task. In this paper, we present the structured segment network (SSN), a novel framework which models the temporal structure of each action instance via a structured temporal pyramid. On top of the pyramid, we further introduce a decomposed discriminative model comprising two classifiers, respectively for classifying actions and determining completeness. This allows the framework to effectively distinguish positive proposals from background or incomplete ones, thus leading to both accurate recognition and localization. These components are integrated into a unified network that can be efficiently trained in an end-to-end fashion. Additionally, a simple yet effective temporal action proposal scheme, dubbed temporal actionness grouping (TAG) is devised to generate high quality action proposals. On two challenging benchmarks, THUMOS14 and ActivityNet, our method remarkably outperforms previous state-of-the-art methods, demonstrating superior accuracy and strong adaptivity in handling actions with various temporal structures.

Key facts. Cited by 267 papers in the map (data through 14 Oct 2026). First posted on arXiv 20 Apr 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 77; 2022: 76; 2023: 42; 2024: 35; 2025: 25; 2026: 12.

267 papers.

Sort:

By area:

  1. 2023VideoComposer: Compositional Video Synthesis with Motion Controllabilitycited by 464
  2. 2022ActionFormer: Localizing Moments of Actions with Transformerscited by 311
  3. 2021Prompting Visual-Language Models for Efficient Video Understandingcited by 310
  4. 2023Vid2Seq: Large-Scale Pretraining of a Visual Language Model for Dense Video Captioningcited by 263
  5. 2024MMBench-Video: A Long-Form Multi-Shot Benchmark for Holistic Video Understandingcited by 182
  6. 2021Learning Salient Boundary Feature for Anchor-free Temporal Action Localizationcited by 156
  7. 2021End-to-end Temporal Action Detection with Transformercited by 151
  8. 2023TriDet: Temporal Action Detection with Relative Boundary Modelingcited by 142
  9. 2021End-to-End Dense Video Captioning with Parallel Decodingcited by 133
  10. 2021Relaxed Transformer Decoders for Direct Action Proposal Generationcited by 117
  11. 2021Temporal Context Aggregation Network for Temporal Action Proposal Refinementcited by 112
  12. 2021Context-aware Biaffine Localizing Network for Temporal Sentence Groundingcited by 107
38 more
  1. 2022FineDiving: A Fine-grained Dataset for Procedure-aware Action Quality Assessmentcited by 107
  2. 2021Negative Sample Matters: A Renaissance of Metric Learning for Temporal Groundingcited by 90
  3. 2021Long Short-Term Transformer for Online Action Detectioncited by 88
  4. 2021MultiSports: A Multi-Person Video Dataset of Spatio-Temporally Localized Sports Actionscited by 84
  5. 2022Temporal Alignment Networks for Long-term Videocited by 84
  6. 2023AV-Deepfake1M: A Large-Scale LLM-Driven Audio-Visual Deepfake Datasetcited by 80
  7. 2022TALLFormer: Temporal Action Localization with a Long-memory Transformercited by 79
  8. 2021Group-aware Contrastive Regression for Action Quality Assessmentcited by 75
  9. 2021Object-Region Video Transformerscited by 73
  10. 2021QVHighlights: Detecting Moments and Highlights in Videos via Natural Language Queriescited by 72
  11. 2024TimeMarker: A Versatile Video-LLM for Long and Short Video Understanding with Superior Temporal Localization Abilitycited by 70
  12. 2021CoLA: Weakly-Supervised Temporal Action Localization with Snippet Contrastive Learningcited by 69
  13. 2023UnLoc: A Unified Framework for Video Localization Taskscited by 66
  14. 2021Generic Event Boundary Detection: A Benchmark for Event Segmentationcited by 65
  15. 2022Zero-Shot Temporal Action Detection via Vision-Language Promptingcited by 65
  16. 2024Number it: Temporal Grounding Videos like Flipping Mangacited by 65
  17. 2021Enriching Local and Global Contexts for Temporal Action Localizationcited by 64
  18. 2021FineAction: A Fine-Grained Video Dataset for Temporal Action Localizationcited by 61
  19. 2022Long-Form Video-Language Pre-Training with Multimodal Temporal Contrastive Learningcited by 59
  20. 2021On Pursuit of Designing Multi-modal Transformer for Video Groundingcited by 54
  21. 2021Target Adaptive Context Aggregation for Video Scene Graph Generationcited by 52
  22. 2021Activity Graph Transformer for Temporal Action Localizationcited by 48
  23. 2022Memory-Guided Semantic Learning Network for Temporal Sentence Groundingcited by 46
  24. 2021A Hybrid Attention Mechanism for Weakly-Supervised Temporal Action Localizationcited by 45
  25. 2023Glitch in the Matrix: A Large Scale Benchmark for Content Driven Audio-Visual Forgery Detection and Localizationcited by 43
  26. 2021Learning Action Completeness from Points for Weakly-supervised Temporal Action Localizationcited by 42
  27. 2022ASM-Loc: Action-aware Segment Modeling for Weakly-Supervised Temporal Action Localizationcited by 40
  28. 2022An Empirical Study of End-to-End Temporal Action Detectioncited by 40
  29. 2021Cross-Sentence Temporal and Semantic Relations in Video Activity Localisationcited by 39
  30. 2021Weakly Supervised Temporal Adjacent Network for Language Groundingcited by 37
  31. 2022Temporal Sentence Grounding in Videos: A Survey and Future Directionscited by 37
  32. 2021Class Semantics-based Attention for Action Detectioncited by 36
  33. 2021DCAN: Improving Temporal Action Detection via Dual Context Aggregationcited by 36
  34. 2021NeRF in detail: Learning to sample for view synthesiscited by 35
  35. 2022Exploring Optical-Flow-Guided Motion and Detection-Based Appearance for Temporal Sentence Groundingcited by 35
  36. 2022Reducing the Vision and Language Bias for Temporal Sentence Groundingcited by 35
  37. 2021Cross-modal Consensus Network for Weakly Supervised Temporal Action Localizationcited by 34
  38. 2022Fine-grained Temporal Contrastive Learning for Weakly-supervised Temporal Action Localizationcited by 33

50 most cited of 267 All 267 (JSON)