mlsanityData to 7 Oct 2026

Learning to Segment Actions from Observation and Narration

arXiv 2005.03684 ↗First posted 7 May 2020cs.CL cs.CVcited by 21

Not on the map. Why?

Abstract (arXiv)

We apply a generative segmental model of task structure, guided by narration, to action segmentation in video. We focus on unsupervised and weakly-supervised settings where no action labels are known during training. Despite its simplicity, our model performs competitively with previous work on a dataset of naturalistic instructional videos. Our model allows us to vary the sources of supervision used in training, and we find that both task structure and narrative language provide large benefits in segmentation quality.

Key facts. Cited by 21 papers in the map (data through 7 Oct 2026). First posted on arXiv 7 May 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 6; 2023: 7; 2024: 2; 2025: 3; 2026: 1.

21 papers.

Sort:

  1. 2024BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructionscited by 527
  2. 2023Grounding Language Models to Images for Multimodal Inputs and Outputscited by 129
  3. 2022Temporal Action Segmentation: An Analysis of Modern Techniquescited by 85
  4. 2022My View is the Best View: Procedure Learning from Egocentric Videoscited by 81
  5. 2023CaptainCook4D: A Dataset for Understanding Errors in Procedural Activitiescited by 47
  6. 2022TL;DW? Summarizing Instructional Videos with Task Relevance & Cross-Modal Saliencycited by 35
  7. 2023Learning to Ground Instructional Articles in Videos through Narrationscited by 28
  8. 2022Boundary-aware Self-supervised Learning for Video Scene Segmentationcited by 17
  9. 2021Temporal Action Segmentation with High-level Complex Activity Labelscited by 16
  10. 2021SVIP: Sequence VerIfication for Procedures in Videoscited by 16
  11. 2024Why Not Use Your Textbook? Knowledge-Enhanced Procedure Planning of Instructional Videoscited by 15
  12. 2023STEPs: Self-Supervised Key Step Extraction and Localization from Unlabeled Procedural Videoscited by 13
9 more
  1. 2023United We Stand, Divided We Fall: UnityGraph for Unsupervised Procedure Learning from Videoscited by 13
  2. 2023LaMPP: Language Models as Probabilistic Priors for Perception and Actioncited by 11
  3. 2022Low-Rank Constraints for Fast Inference in Structured Modelscited by 8
  4. 2022LiveSeg: Unsupervised Multimodal Temporal Segmentation of Long Livestream Videoscited by 6
  5. 2025Procedure Learning via Regularized Gromov-Wasserstein Optimal Transportcited by 4
  6. 2025REMAP: Regularized Matching and Partial Alignment of Video Embeddingscited by 1
  7. 2026Hierarchical Action Learning for Weakly-Supervised Action Segmentationcited by 1
  8. 2023BIT: Bi-Level Temporal Modeling for Efficient Supervised Action Segmentationcited by 0
  9. 2025I2G: Generating Instructional Illustrations via Text-Conditioned Diffusioncited by 0