An end-to-end generative framework for video segmentation and recognition
Not on the map. Why?
Abstract (arXiv)
We describe an end-to-end generative approach for the segmentation and recognition of human activities. In this approach, a visual representation based on reduced Fisher Vectors is combined with a structured temporal model for recognition. We show that the statistical properties of Fisher Vectors make them an especially suitable front-end for generative models such as Gaussian mixtures. The system is evaluated for both the recognition of complex activities as well as their parsing into action units. Using a variety of video datasets ranging from human cooking activities to animal behaviors, our experiments demonstrate that the resulting architecture outperforms state-of-the-art approaches for larger datasets, i.e. when sufficient amount of data is available for training structured generative models.
Key facts. Cited by 61 papers in the map (data through 14 Oct 2026). First posted on arXiv 7 Sep 2015.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 20; 2022: 12; 2023: 12; 2024: 6; 2025: 7; 2026: 4.
61 papers.
Sort:
By area:
- 2021ASFormer: Transformer for Action Segmentationcited by 168
- 2022Temporal Action Segmentation: An Analysis of Modern Techniquescited by 85
- 2021Generic Event Boundary Detection: A Benchmark for Event Segmentationcited by 65
- 2021Temporally-Weighted Hierarchical Clustering for Unsupervised Action Segmentationcited by 58
- 2022Unified Fully and Timestamp Supervised Temporal Action Segmentation via Sequence to Sequence Translationcited by 55
- 2021Unsupervised Action Segmentation by Joint Representation Learning and Online Clusteringcited by 49
- 2021Temporal Action Segmentation from Timestamp Supervisioncited by 48
- 2021Global2Local: Efficient Structure Search for Video Action Segmentationcited by 42
- 2023How Much Temporal Long-Term Context is Needed for Action Segmentation?cited by 36
- 2021Action Shuffle Alternating Learning for Unsupervised Action Segmentationcited by 33
- 2021Coarse to Fine Multi-Resolution Temporal Convolutional Networkcited by 28
- 2023Learning to Ground Instructional Articles in Videos through Narrationscited by 28
38 more
- 2021Unsupervised Discriminative Embedding for Sub-Action Learning in Complex Activitiescited by 26
- 2021Action Segmentation with Mixed Temporal Domain Adaptationcited by 21
- 2024Hierarchical Vector Quantization for Unsupervised Action Segmentationcited by 19
- 2022Timestamp-Supervised Action Segmentation with Graph Convolutional Networkscited by 18
- 2023Permutation-Aware Action Segmentation via Unsupervised Frame-to-Segment Alignmentcited by 17
- 2021FIFA: Fast Inference Approximation for Action Segmentationcited by 16
- 2021Temporal Action Segmentation with High-level Complex Activity Labelscited by 16
- 2021SSCAP: Self-supervised Co-occurrence Action Parsing for Unsupervised Temporal Action Segmentationcited by 15
- 2022Cross-Enhancement Transformer for Action Segmentationcited by 14
- 2024FineBio: A Fine-Grained Video Dataset of Biological Experiments with Hierarchical Annotationcited by 12
- 2021Hierarchical Modeling for Task Recognition and Action Segmentation in Weakly-Labeled Instructional Videoscited by 11
- 2023OTAS: Unsupervised Boundary Detection for Object-Centric Temporal Action Segmentationcited by 11
- 2024Step Differences in Instructional Videocited by 11
- 2021Automated freezing of gait assessment with marker-based motion capture and multi-stage spatial-temporal graph convolutional neural networkscited by 10
- 2021Sign Segmentation with Changepoint-Modulated Pseudo-Labellingcited by 10
- 2022Do we really need temporal convolutions in action segmentation?cited by 10
- 2022RF-Next: Efficient Receptive Field Search for Convolutional Neural Networkscited by 10
- 2023Leveraging triplet loss for unsupervised action segmentationcited by 10
- 2021Anchor-Constrained Viterbi for Set-Supervised Action Segmentationcited by 8
- 2021CoSeg: Cognitively Inspired Unsupervised Generic Event Segmentationcited by 7
- 2025F3Set: Towards Analyzing Fast, Frequent, and Fine-grained Events from Videoscited by 7
- 2022A-ACT: Action Anticipation through Cycle Transformationscited by 6
- 2022Robust Action Segmentation from Timestamp Supervisioncited by 6
- 2025Skeleton Motion Words for Unsupervised Skeleton-Based Temporal Action Segmentationcited by 6
- 2021Review of Video Predictive Understanding: Early Action Recognition and Future Action Predictioncited by 5
- 2022A Circular Window-based Cascade Transformer for Online Action Detectioncited by 4
- 2022Timestamp-Supervised Action Segmentation from the Perspective of Clusteringcited by 4
- 2023Prompt-enhanced Hierarchical Transformer Elevating Cardiopulmonary Resuscitation Instruction via Temporal Action Segmentationcited by 4
- 2023Temporal Segment Transformer for Action Segmentationcited by 3
- 2026HOI-aware Adaptive Network for Weakly-supervised Action Segmentationcited by 3
- 2023DIR-AS: Decoupling Individual Identification and Temporal Reasoning for Action Segmentationcited by 2
- 2024ViTALS: Vision Transformer for Action Localization in Surgical Nephrectomycited by 2
- 2024Faster Diffusion Action Segmentationcited by 2
- 2025Pose-Aware Weakly-Supervised Action Segmentationcited by 2
- 2021Sequence-to-Sequence Modeling for Action Identification at High Temporal Resolutioncited by 1
- 2023Is there progress in activity progress prediction?cited by 1
- 2023SMC-NCA: Semantic-guided Multi-level Contrast for Semi-supervised Temporal Action Segmentationcited by 1
- 2024Temporal2Seq: A Unified Framework for Temporal Video Understanding Taskscited by 1
50 most cited of 61 All 61 (JSON)
- 2026Improving Temporal Action Segmentation via Constraint-Aware Decoding11 May 2026
- 2026HOI-aware Adaptive Network for Weakly-supervised Action Segmentation29 Apr 2026
- 2026Combining Boundary Supervision and Segment-Level Regularization for Fine-Grained Action Segmentation2 Apr 2026
- 2026Hierarchical Action Learning for Weakly-Supervised Action Segmentation27 Feb 2026
- 2025Knowledge Graph Completion for Action Prediction on Situational Graphs -- A Case Study on Household Tasks19 Aug 2025
- 2025Looking into the Unknown: Exploring Action Discovery for Segmentation of Known and Unknown Actions7 Aug 2025
- 2025Skeleton Motion Words for Unsupervised Skeleton-Based Temporal Action Segmentation6 Aug 2025
- 2025F3Set: Towards Analyzing Fast, Frequent, and Fine-grained Events from Videos11 Apr 2025
- 2025Pose-Aware Weakly-Supervised Action Segmentation8 Apr 2025
- 2025Towards Generalizing Temporal Action Segmentation to Unseen Views3 Apr 2025
- 2025Improving action segmentation via explicit similarity measurement15 Feb 2025
- 2024Hierarchical Vector Quantization for Unsupervised Action Segmentation23 Dec 2024
38 more
- 2024Temporal2Seq: A Unified Framework for Temporal Video Understanding Tasks27 Sep 2024
- 2024Faster Diffusion Action Segmentation4 Aug 2024
- 2024ViTALS: Vision Transformer for Action Localization in Surgical Nephrectomy4 May 2024
- 2024Step Differences in Instructional Video24 Apr 2024
- 2024FineBio: A Fine-Grained Video Dataset of Biological Experiments with Hierarchical Annotation1 Feb 2024
- 2023SMC-NCA: Semantic-guided Multi-level Contrast for Semi-supervised Temporal Action Segmentation19 Dec 2023
- 2023OTAS: Unsupervised Boundary Detection for Object-Centric Temporal Action Segmentation12 Sep 2023
- 2023Human Action Co-occurrence in Lifestyle Vlogs using Graph Link Prediction12 Sep 2023
- 2023Prompt-enhanced Hierarchical Transformer Elevating Cardiopulmonary Resuscitation Instruction via Temporal Action Segmentation31 Aug 2023
- 2023BIT: Bi-Level Temporal Modeling for Efficient Supervised Action Segmentation28 Aug 2023
- 2023How Much Temporal Long-Term Context is Needed for Action Segmentation?22 Aug 2023
- 2023Is there progress in activity progress prediction?10 Aug 2023
- 2023Learning to Ground Instructional Articles in Videos through Narrations6 Jun 2023
- 2023Permutation-Aware Action Segmentation via Unsupervised Frame-to-Segment Alignment31 May 2023
- 2023Leveraging triplet loss for unsupervised action segmentation13 Apr 2023
- 2023DIR-AS: Decoupling Individual Identification and Temporal Reasoning for Action Segmentation4 Apr 2023
- 2023Temporal Segment Transformer for Action Segmentation25 Feb 2023
- 2022Timestamp-Supervised Action Segmentation from the Perspective of Clustering22 Dec 2022
- 2022Facial Tic Detection in Untrimmed Videos of Tourette Syndrome Patients7 Nov 2022
- 2022Distill and Collect for Semi-Supervised Temporal Action Segmentation2 Nov 2022
- 2022Temporal Action Segmentation: An Analysis of Modern Techniques19 Oct 2022
- 2022Robust Action Segmentation from Timestamp Supervision12 Oct 2022
- 2022Unified Fully and Timestamp Supervised Temporal Action Segmentation via Sequence to Sequence Translation1 Sep 2022
- 2022A Circular Window-based Cascade Transformer for Online Action Detection30 Aug 2022
- 2022Timestamp-Supervised Action Segmentation with Graph Convolutional Networks30 Jun 2022
- 2022RF-Next: Efficient Receptive Field Search for Convolutional Neural Networks14 Jun 2022
- 2022Do we really need temporal convolutions in action segmentation?26 May 2022
- 2022Cross-Enhancement Transformer for Action Segmentation19 May 2022
- 2022A-ACT: Action Anticipation through Cycle Transformations2 Apr 2022
- 2021Sequence-to-Sequence Modeling for Action Identification at High Temporal Resolution3 Nov 2021
- 2021ASFormer: Transformer for Action Segmentation16 Oct 2021
- 2021Hierarchical Modeling for Task Recognition and Action Segmentation in Weakly-Labeled Instructional Videos12 Oct 2021
- 2021CoSeg: Cognitively Inspired Unsupervised Generic Event Segmentation30 Sep 2021
- 2021Temporal Action Segmentation with High-level Complex Activity Labels15 Aug 2021
- 2021FIFA: Fast Inference Approximation for Action Segmentation9 Aug 2021
- 2021Review of Video Predictive Understanding: Early Action Recognition and Future Action Prediction11 Jul 2021
- 2021SSCAP: Self-supervised Co-occurrence Action Parsing for Unsupervised Temporal Action Segmentation29 May 2021
- 2021Unsupervised Action Segmentation by Joint Representation Learning and Online Clustering27 May 2021
50 newest of 61 All 61 (JSON)
- 2024Faster Diffusion Action Segmentationcited by 2
- 2024Faster Diffusion Action Segmentation4 Aug 2024
- 2021ASFormer: Transformer for Action Segmentationcited by 168
- 2022Temporal Action Segmentation: An Analysis of Modern Techniquescited by 85
- 2021Generic Event Boundary Detection: A Benchmark for Event Segmentationcited by 65
- 2021Temporally-Weighted Hierarchical Clustering for Unsupervised Action Segmentationcited by 58
- 2022Unified Fully and Timestamp Supervised Temporal Action Segmentation via Sequence to Sequence Translationcited by 55
- 2021Unsupervised Action Segmentation by Joint Representation Learning and Online Clusteringcited by 49
- 2021Temporal Action Segmentation from Timestamp Supervisioncited by 48
- 2021Global2Local: Efficient Structure Search for Video Action Segmentationcited by 42
- 2023How Much Temporal Long-Term Context is Needed for Action Segmentation?cited by 36
- 2021Action Shuffle Alternating Learning for Unsupervised Action Segmentationcited by 33
- 2021Coarse to Fine Multi-Resolution Temporal Convolutional Networkcited by 28
- 2023Learning to Ground Instructional Articles in Videos through Narrationscited by 28
12 most cited of 60 All 60 (JSON)
- 2026Improving Temporal Action Segmentation via Constraint-Aware Decoding11 May 2026
- 2026HOI-aware Adaptive Network for Weakly-supervised Action Segmentation29 Apr 2026
- 2026Combining Boundary Supervision and Segment-Level Regularization for Fine-Grained Action Segmentation2 Apr 2026
- 2026Hierarchical Action Learning for Weakly-Supervised Action Segmentation27 Feb 2026
- 2025Knowledge Graph Completion for Action Prediction on Situational Graphs -- A Case Study on Household Tasks19 Aug 2025
- 2025Looking into the Unknown: Exploring Action Discovery for Segmentation of Known and Unknown Actions7 Aug 2025
- 2025Skeleton Motion Words for Unsupervised Skeleton-Based Temporal Action Segmentation6 Aug 2025
- 2025F3Set: Towards Analyzing Fast, Frequent, and Fine-grained Events from Videos11 Apr 2025
- 2025Pose-Aware Weakly-Supervised Action Segmentation8 Apr 2025
- 2025Towards Generalizing Temporal Action Segmentation to Unseen Views3 Apr 2025
- 2025Improving action segmentation via explicit similarity measurement15 Feb 2025
- 2024Hierarchical Vector Quantization for Unsupervised Action Segmentation23 Dec 2024
12 newest of 60 All 60 (JSON)