# All Eyes on the Workflow: Automated and Efficient Event Discovery from Video Streams

arXiv: https://arxiv.org/abs/2604.22476 · HTML: https://mlsanity.com/paper/2604.22476/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 29 references that are in the map.
- First posted on arXiv 24 Apr 2026.

## Builds on (most cited first, 19 of 19)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [A Closer Look at Spatiotemporal Convolutions for Action Recognition](https://mlsanity.com/paper/1711.11248/) (2017)
- [Do Vision Transformers See Like Convolutional Neural Networks?](https://mlsanity.com/paper/2108.08810/) (2021)
- [Rescaling Egocentric Vision](https://mlsanity.com/paper/2006.13256/) (2020)
- [Self-Supervised Learning for Videos: A Survey](https://mlsanity.com/paper/2207.00419/) (2022)
- [Temporal Action Segmentation: An Analysis of Modern Techniques](https://mlsanity.com/paper/2210.10352/) (2022)
- [Foundation Models for Video Understanding: A Survey](https://mlsanity.com/paper/2405.03770/) (2024)
- [Zero-shot Natural Language Video Localization](https://mlsanity.com/paper/2110.00428/) (2021)
- [Conformance Checking over Uncertain Event Data](https://mlsanity.com/paper/2009.14452/) (2020)
- [A Comprehensive Review of Few-shot Action Recognition](https://mlsanity.com/paper/2407.14744/) (2024)
- [Uncertain Process Data with Probabilistic Knowledge: Problem Characterization and Challenges](https://mlsanity.com/paper/2106.03324/) (2021)
- [Generalized Few-Shot Video Classification with Video Retrieval and Feature Generation](https://mlsanity.com/paper/2007.04755/) (2020)
- [Shedding Light on Blind Spots: Developing a Reference Architecture to Leverage Video Data for Process Mining](https://mlsanity.com/paper/2010.11289/) (2020)
- [A Survey on Deep Learning-based Spatio-temporal Action Detection](https://mlsanity.com/paper/2308.01618/) (2023)
- [Process Mining for Unstructured Data: Challenges and Research Directions](https://mlsanity.com/paper/2401.13677/) (2023)
- [Probabilistic and Non-Deterministic Event Data in Process Mining: Embedding Uncertainty in Process Analysis Techniques](https://mlsanity.com/paper/2205.04827/) (2022)
- [Few-Shot Learning in Video and 3D Object Detection: A Survey](https://mlsanity.com/paper/2507.17079/) (2025)
- [Triples and Knowledge-Infused Embeddings for Clustering and Classification of Scientific Documents](https://mlsanity.com/paper/2601.08841/) (2025)

## Abstract (arXiv)

Disciplines such as business process management and process mining aid organizations by discovering insights about processes on the basis of recorded event data. However, an obstacle to process analysis is data multi-modality: for instance, data in video form are not directly interpretable as events. Existing approaches rely on a dictionary of activity label as input, cannot provide frame-by-frame labeling explanations, or rely on superseded computer vision techniques. In this work, we present SnapLog, an approach to extract event data from videos by converting frames to feature vectors using image embeddings and performing temporal segmentation through frame-wise similarity matrices. A generalized few-shot classification is then used to assign labels to the video segments, yielding labeled, timestamped sub-sequences of frames that are interpretable as events. Conventional process mining techniques can be used to analyze the resulting data. We show that our approach produces logs that accurately reflect the process in the videos.