mlsanityData to 14 Oct 2026

End-to-End Action Segmentation Transformer

arXiv 2503.06316 ↗First posted 8 Mar 2025cs.CV eess.IVcited by 1

Not on the map. Why?

Abstract (arXiv)

Most recent work on action segmentation relies on pre-computed frame features from models trained on other tasks and typically focuses on framewise encoding and labeling without explicitly modeling action segments. To overcome these limitations, we introduce the End-to-End Action Segmentation Transformer (EAST), which processes raw video frames directly -- eliminating the need for pre-extracted features and enabling true end-to-end training. Our contributions are as follows: (1) a lightweight adapter design for effective fine-tuning of large backbones; (2) an efficient segmentation-by-detection framework for leveraging action proposals predicted over a coarsely downsampled video; and (3) a novel action-proposal-based data augmentation strategy. EAST achieves SOTA performance on standard benchmarks, including GTEA, 50Salads, Breakfast, and Assembly-101.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 28 of its 44 references that are in the map. First posted on arXiv 8 Mar 2025.

Builds on

28 of 44 references.

References · 28

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  2. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  3. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
  4. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,845
  5. 2016Gaussian Error Linear Units (GELUs)cited by 3,502
  6. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,270
  7. 2017The Kinetics Human Action Video Datasetcited by 1,909
  8. 2019FCOS: Fully Convolutional One-Stage Object Detectioncited by 1,577
  9. 2022VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Trainingcited by 1,322
  10. 2021Scaling Vision Transformerscited by 999
  11. 2018TSM: Temporal Shift Module for Efficient Video Understandingcited by 856
  12. 2023VideoMAE V2: Scaling Video Masked Autoencoders with Dual Maskingcited by 528
16 more
  1. 2016Temporal Convolutional Networks for Action Segmentation and Detectioncited by 508
  2. 2019Distance-IoU Loss: Faster and Better Learning for Bounding Box Regressioncited by 459
  3. 2019MS-TCN: Multi-Stage Temporal Convolutional Network for Action Segmentationcited by 373
  4. 2022ActionFormer: Localizing Moments of Actions with Transformerscited by 311
  5. 2022Assembly101: A Large-Scale Multi-View Video Dataset for Understanding Procedural Activitiescited by 287
  6. 2021ASFormer: Transformer for Action Segmentationcited by 168
  7. 2022Temporal Action Segmentation: An Analysis of Modern Techniquescited by 85
  8. 2023Diffusion Action Segmentationcited by 74
  9. 2022Unified Fully and Timestamp Supervised Temporal Action Segmentation via Sequence to Sequence Translationcited by 55
  10. 2022Bridge-Prompt: Towards Ordinal Action Understanding in Instructional Videoscited by 50
  11. 2023End-to-End Temporal Action Detection with 1B Parameters Across 1000 Framescited by 47
  12. 2023How Much Temporal Long-Term Context is Needed for Action Segmentation?cited by 36
  13. 2021Action Shuffle Alternating Learning for Unsupervised Action Segmentationcited by 33
  14. 2021Coarse to Fine Multi-Resolution Temporal Convolutional Networkcited by 28
  15. 2020MS-TCN++: Multi-Stage Temporal Convolutional Network for Action Segmentationcited by 19
  16. 2024Efficient Temporal Action Segmentation via Boundary-aware Query Votingcited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Polyphony: Diffusion-based Dual-Hand Action Segmentation with Alternating Vision Transformer and Semantic Conditioningcited by 1