mlsanityData to 7 Oct 2026

Global2Local: Efficient Structure Search for Video Action Segmentation

arXiv 2101.00910 ↗First posted 4 Jan 2021cs.CVcited by 42

Not on the map. Why?

Abstract (arXiv)

Temporal receptive fields of models play an important role in action segmentation. Large receptive fields facilitate the long-term relations among video clips while small receptive fields help capture the local details. Existing methods construct models with hand-designed receptive fields in layers. Can we effectively search for receptive field combinations to replace hand-designed patterns? To answer this question, we propose to find better receptive field combinations through a global-to-local search scheme. Our search scheme exploits both global search to find the coarse combinations and local search to get the refined receptive field combination patterns further. The global search finds possible coarse combinations other than human-designed patterns. On top of the global search, we propose an expectation guided iterative local search scheme to refine combinations effectively. Our global-to-local search can be plugged into existing action segmentation methods to achieve state-of-the-art performance.

Key facts. Cited by 42 papers in the map (data through 7 Oct 2026). Builds on 36 of its 68 references that are in the map. First posted on arXiv 4 Jan 2021.

Builds on

36 of 68 references.

References · 36

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  4. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  5. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
  6. 2019Searching for MobileNetV3cited by 1,984
  7. 2018SlowFast Networks for Video Recognitioncited by 1,862
  8. 2018DARTS: Differentiable Architecture Searchcited by 1,782
  9. 2014Two-Stream Convolutional Networks for Action Recognition in Videoscited by 1,067
  10. 2018Regularized Evolution for Image Classifier Architecture Searchcited by 902
  11. 2018ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardwarecited by 666
  12. 2016Temporal Convolutional Networks for Action Segmentation and Detectioncited by 504
24 more
  1. 2019Res2Net: A New Multi-scale Backbone Architecturecited by 468
  2. 2019MS-TCN: Multi-Stage Temporal Convolutional Network for Action Segmentationcited by 370
  3. 2019Auto-DeepLab: Hierarchical Neural Architecture Search for Semantic Image Segmentationcited by 258
  4. 2017Hierarchical Representations for Efficient Architecture Searchcited by 188
  5. 2019Understanding and Robustifying Differentiable Architecture Searchcited by 176
  6. 2017Genetic CNNcited by 124
  7. 2020Highly Efficient Salient Object Detection with 100K Parameterscited by 79
  8. 2016Segmental Spatiotemporal CNNs for Fine-grained Action Segmentationcited by 78
  9. 2018Weakly-Supervised Action Segmentation with Iterative Soft Boundary Assignmentcited by 74
  10. 2020Action Segmentation with Joint Self-Supervised Temporal Domain Adaptationcited by 74
  11. 2017Weakly Supervised Action Learning with RNN based Fine-to-coarse Modelingcited by 70
  12. 2019Probabilistic Forecasting with Temporal Convolutional Neural Networkcited by 63
  13. 2015An end-to-end generative framework for video segmentation and recognitioncited by 61
  14. 2018Unsupervised Learning and Segmentation of Complex Activities from Videocited by 61
  15. 2018Automatically designing CNN architectures using genetic algorithm for image classificationcited by 48
  16. 2016Weakly supervised learning of actions from transcriptscited by 46
  17. 2020Deep Hough Transform for Semantic Line Detectioncited by 42
  18. 2020SCT: Set Constrained Temporal Transformer for Set Supervised Action Segmentationcited by 39
  19. 2019A Hybrid RNN-HMM Approach for Weakly Supervised Temporal Action Segmentationcited by 36
  20. 2020Set-Constrained Viterbi for Set-Supervised Action Segmentationcited by 35
  21. 2020DOTS: Decoupling Operation and Topology in Differentiable Architecture Searchcited by 23
  22. 2020MS-TCN++: Multi-Stage Temporal Convolutional Network for Action Segmentationcited by 18
  23. 2017TricorNet: A Hybrid Temporal Convolutional and Recurrent Network for Video Action Segmentationcited by 14
  24. 2019Coupled Generative Adversarial Network for Continuous Fine-grained Action Segmentationcited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 13; 2023: 15; 2024: 5; 2025: 4; 2026: 3.

42 papers.

Sort:

By area:

  1. 2021Concealed Object Detectioncited by 212
  2. 2021ASFormer: Transformer for Action Segmentationcited by 166
  3. 2022Temporal Action Segmentation: An Analysis of Modern Techniquescited by 85
  4. 2023Diffusion Action Segmentationcited by 74
  5. 2022Unified Fully and Timestamp Supervised Temporal Action Segmentation via Sequence to Sequence Translationcited by 55
  6. 2022Bridge-Prompt: Towards Ordinal Action Understanding in Instructional Videoscited by 50
  7. 2023VidChapters-7M: Video Chapters at Scalecited by 39
  8. 2023How Much Temporal Long-Term Context is Needed for Action Segmentation?cited by 36
  9. 2022C2F-TCN: A Framework for Semi and Fully Supervised Temporal Action Segmentationcited by 23
  10. 2024Efficient and Effective Weakly-Supervised Action Segmentation via Action-Transition-Aware Boundary Alignmentcited by 13
  11. 2022Weakly-Supervised Online Action Segmentation in Multi-View Instructional Videoscited by 12
  12. 2022Continuous Human Action Recognition for Human-Machine Interaction: A Reviewcited by 10
30 more
  1. 2022Do we really need temporal convolutions in action segmentation?cited by 10
  2. 2022RF-Next: Efficient Receptive Field Search for Convolutional Neural Networkscited by 10
  3. 2024Efficient Temporal Action Segmentation via Boundary-aware Query Votingcited by 10
  4. 2022Semantic2Graph: Graph-based Multi-modal Feature Fusion for Action Segmentation in Videoscited by 9
  5. 2023Co-attention Propagation Network for Zero-Shot Video Object Segmentationcited by 9
  6. 2025Chapter-Llama: Efficient Chaptering in Hour-Long Videos with LLMscited by 9
  7. 2023Activity Grammars for Temporal Action Segmentationcited by 8
  8. 2023A Decoupled Spatio-Temporal Framework for Skeleton-based Action Segmentationcited by 7
  9. 2022Robust Action Segmentation from Timestamp Supervisioncited by 6
  10. 2023SF-TMN: SlowFast Temporal Modeling Network for Surgical Phase Recognitioncited by 5
  11. 2024ActFusion: a Unified Diffusion Model for Action Segmentation and Anticipationcited by 4
  12. 2023Temporal Segment Transformer for Action Segmentationcited by 3
  13. 2023Enhancing Transformer Backbone for Egocentric Video Action Segmentationcited by 3
  14. 2024Long-Tail Temporal Action Segmentation with Group-wise Temporal Logit Adjustmentcited by 3
  15. 2022Hand Hygiene Assessment via Joint Step Segmentation and Key Action Scorercited by 2
  16. 2022Cross Task Neural Architecture Search for EEG Signal Classificationscited by 2
  17. 2023TAEC: Unsupervised Action Segmentation with Temporal-Aware Embedding and Clusteringcited by 2
  18. 2023DIR-AS: Decoupling Individual Identification and Temporal Reasoning for Action Segmentationcited by 2
  19. 2024Language-Assisted Human Part Motion Learning for Skeleton-Based Temporal Action Segmentationcited by 2
  20. 2022A Generalized & Robust Framework For Timestamp Supervision in Temporal Action Segmentationcited by 1
  21. 2023Latent Graph Attention for Enhanced Spatial Contextcited by 1
  22. 2023End-to-End Streaming Video Temporal Action Segmentation with Reinforce Learningcited by 1
  23. 2023SigFormer: Sparse Signal-Guided Transformer for Multi-Modal Human Action Segmentationcited by 1
  24. 2025Localization-Aware Multi-Scale Representation Learning for Repetitive Action Countingcited by 1
  25. 2025Dual-Stream Alignment for Action Segmentationcited by 1
  26. 2023Subtle Signals: Video-based Detection of Infant Non-nutritive Sucking as a Neurodevelopmental Cuecited by 0
  27. 2025Cost-Sensitive Learning for Long-Tailed Temporal Action Segmentationcited by 0
  28. 2026Learning Action Hierarchies via Hybrid Geometric Diffusioncited by 0
  29. 2026LaDy: Lagrangian-Dynamic Informed Network for Skeleton-based Action Segmentation via Spatial-Temporal Modulationcited by 0
  30. 2026Spectral Scalpel: Amplifying Adjacent Action Discrepancy via Frequency-Selective Filtering for Skeleton-Based Action Segmentationcited by 0