# Anomalous Frame Detection by Grouping Frame Similarities between Two Videos Computed by Vision-Language Model to Extract Expert Workers' Unique Actions

arXiv: https://arxiv.org/abs/2607.10598 · HTML: https://mlsanity.com/paper/2607.10598/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 27 references that are in the map.
- First posted on arXiv 12 Jul 2026.

## Builds on (most cited first, 12 of 12)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [ImageBind: One Embedding Space To Bind Them All](https://mlsanity.com/paper/2305.05665/) (2023)
- [ActionCLIP: A New Paradigm for Video Action Recognition](https://mlsanity.com/paper/2109.08472/) (2021)
- [InternVideo2: Scaling Foundation Models for Multimodal Video Understanding](https://mlsanity.com/paper/2403.15377/) (2024)
- [Video Understanding with Large Language Models: A Survey](https://mlsanity.com/paper/2312.17432/) (2023)
- [VideoLLM: Modeling Video Sequence with Large Language Models](https://mlsanity.com/paper/2305.13292/) (2023)
- [Harnessing Large Language Models for Training-free Video Anomaly Detection](https://mlsanity.com/paper/2404.01014/) (2024)
- [Video Similarity and Alignment Learning on Partial Video Copy Detection](https://mlsanity.com/paper/2108.01817/) (2021)
- [Self-Supervised Video Similarity Learning](https://mlsanity.com/paper/2304.03378/) (2023)
- [Step Differences in Instructional Video](https://mlsanity.com/paper/2404.16222/) (2024)
- [Learning Segment Similarity and Alignment in Large-Scale Content Based Video Retrieval](https://mlsanity.com/paper/2309.11091/) (2023)

## Abstract (arXiv)

Maintenance of critical infrastructures, such as railways and power plants, is essential for operational safety and reliability. However, the declining number of skilled maintenance workers poses a serious challenge to sustaining these operations, highlighting the need to effectively transfer expert know-how to less experienced workers. Although traditional interview-based approaches have been used to elicit maintenance skills, they struggle to capture know-how that experts themselves may not consciously recognize. To address this gap, we proposed a method that detects anomalous frames of candidate actions including know-how by comparing a video of manual-based work with that of expert maintenance workers. In a simulated maintenance experiment involving a distribution board, our method targeted 11 types of actions not described in the manual and achieved a 66.9% extraction rate, marking a 50-percentage-point improvement over conventional techniques. These findings underscore the effectiveness of our approach in revealing hidden maintenance knowledge, thereby contributing to enhanced skill transfer and workforce development in critical infrastructure maintenance.