# NeuralNetwork-Viterbi: A Framework for Weakly Supervised Video Learning

arXiv: https://arxiv.org/abs/1805.06875 · HTML: https://mlsanity.com/paper/1805.06875/

## Key facts

- Cited by 67 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 May 2018.

## Built on by (most cited first, 30 of 67)

- [VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understanding](https://mlsanity.com/paper/2109.14084/) (2021)
- [VLM: Task-agnostic Video-Language Model Pre-training for Video Understanding](https://mlsanity.com/paper/2105.09996/) (2021)
- [TACo: Token-aware Cascade Contrastive Learning for Video-Text Alignment](https://mlsanity.com/paper/2108.09980/) (2021)
- [Temporal Action Segmentation: An Analysis of Modern Techniques](https://mlsanity.com/paper/2210.10352/) (2022)
- [Temporal Alignment Networks for Long-term Video](https://mlsanity.com/paper/2204.02968/) (2022)
- [My View is the Best View: Procedure Learning from Egocentric Videos](https://mlsanity.com/paper/2207.10883/) (2022)
- [Temporally-Weighted Hierarchical Clustering for Unsupervised Action Segmentation](https://mlsanity.com/paper/2103.11264/) (2021)
- [Unified Fully and Timestamp Supervised Temporal Action Segmentation via Sequence to Sequence Translation](https://mlsanity.com/paper/2209.00638/) (2022)
- [P3IV: Probabilistic Procedure Planning from Instructional Videos with Weak Supervision](https://mlsanity.com/paper/2205.02300/) (2022)
- [Unsupervised Action Segmentation by Joint Representation Learning and Online Clustering](https://mlsanity.com/paper/2105.13353/) (2021)
- [Temporal Action Segmentation from Timestamp Supervision](https://mlsanity.com/paper/2103.06669/) (2021)
- [Drop-DTW: Aligning Common Signal Between Sequences While Dropping Outliers](https://mlsanity.com/paper/2108.11996/) (2021)
- [StepFormer: Self-supervised Step Discovery and Localization in Instructional Videos](https://mlsanity.com/paper/2304.13265/) (2023)
- [Action Shuffle Alternating Learning for Unsupervised Action Segmentation](https://mlsanity.com/paper/2104.02116/) (2021)
- [Learning to Ground Instructional Articles in Videos through Narrations](https://mlsanity.com/paper/2306.03802/) (2023)
- [Unsupervised Discriminative Embedding for Sub-Action Learning in Complex Activities](https://mlsanity.com/paper/2105.00067/) (2021)
- [SCHEMA: State CHangEs MAtter for Procedure Planning in Instructional Videos](https://mlsanity.com/paper/2403.01599/) (2024)
- [Learning Discriminative Prototypes with Dynamic Time Warping](https://mlsanity.com/paper/2103.09458/) (2021)
- [C2F-TCN: A Framework for Semi and Fully Supervised Temporal Action Segmentation](https://mlsanity.com/paper/2212.11078/) (2022)
- [Iterative Contrast-Classify For Semi-supervised Temporal Action Segmentation](https://mlsanity.com/paper/2112.01402/) (2021)
- [Hierarchical Vector Quantization for Unsupervised Action Segmentation](https://mlsanity.com/paper/2412.17640/) (2024)
- [Timestamp-Supervised Action Segmentation with Graph Convolutional Networks](https://mlsanity.com/paper/2206.15031/) (2022)
- [Permutation-Aware Action Segmentation via Unsupervised Frame-to-Segment Alignment](https://mlsanity.com/paper/2305.19478/) (2023)
- [FIFA: Fast Inference Approximation for Action Segmentation](https://mlsanity.com/paper/2108.03894/) (2021)
- [Temporal Action Segmentation with High-level Complex Activity Labels](https://mlsanity.com/paper/2108.06706/) (2021)
- [SSCAP: Self-supervised Co-occurrence Action Parsing for Unsupervised Temporal Action Segmentation](https://mlsanity.com/paper/2105.14158/) (2021)
- [LASER: A Neuro-Symbolic Framework for Learning Spatial-Temporal Scene Graphs with Weak Supervision](https://mlsanity.com/paper/2304.07647/) (2023)
- [United We Stand, Divided We Fall: UnityGraph for Unsupervised Procedure Learning from Videos](https://mlsanity.com/paper/2311.03550/) (2023)
- [Efficient and Effective Weakly-Supervised Action Segmentation via Action-Transition-Aware Boundary Alignment](https://mlsanity.com/paper/2403.19225/) (2024)
- [Causal Temporal Representation Learning with Nonstationary Sparse Transition](https://mlsanity.com/paper/2409.03142/) (2024)

## Abstract (arXiv)

Video learning is an important task in computer vision and has experienced increasing interest over the recent years. Since even a small amount of videos easily comprises several million frames, methods that do not rely on a frame-level annotation are of special importance. In this work, we propose a novel learning algorithm with a Viterbi-based loss that allows for online and incremental learning of weakly annotated video data. We moreover show that explicit context and length modeling leads to huge improvements in video segmentation and labeling tasks andinclude these models into our framework. On several action segmentation benchmarks, we obtain an improvement of up to 10% compared to current state-of-the-art methods.