# Cascading multi-agent anomaly detection in surveillance systems via vision-language models and embedding-based classification

arXiv: https://arxiv.org/abs/2601.06204 · HTML: https://mlsanity.com/paper/2601.06204/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Chart understanding and anomaly detection, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 7 of its 19 references that are in the map.
- First posted on arXiv 8 Jan 2026.

## Placed in

- Multimodal LLMs / vision-language models / [Chart understanding and anomaly detection](https://mlsanity.com/mllm/chart-understanding-and-anomaly-detection/)

## Builds on (most cited first, 7 of 7)

- [LLaVA-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Models](https://mlsanity.com/paper/2407.07895/) (2024)
- [YOLOv9: Learning What You Want to Learn Using Programmable Gradient Information](https://mlsanity.com/paper/2402.13616/) (2024)
- [VadCLIP: Adapting Vision-Language Models for Weakly Supervised Video Anomaly Detection](https://mlsanity.com/paper/2308.11681/) (2023)
- [Deep Learning for Video Anomaly Detection: A Review](https://mlsanity.com/paper/2409.05383/) (2024)
- [Networking Systems for Video Anomaly Detection: A Tutorial and Survey](https://mlsanity.com/paper/2405.10347/) (2024)
- [Privacy-Preserving Video Anomaly Detection: A Survey](https://mlsanity.com/paper/2411.14565/) (2024)
- [Vision-Language Models Assisted Unsupervised Video Anomaly Detection](https://mlsanity.com/paper/2409.14109/) (2024)

## Abstract (arXiv)

Intelligent anomaly detection in dynamic visual environments requires reconciling real-time performance with semantic interpretability. Conventional approaches address only fragments of this challenge. Reconstruction-based models capture low-level deviations without contextual reasoning, object detectors provide speed but limited semantics, and large vision-language systems deliver interpretability at prohibitive computational cost. This work introduces a cascading multi-agent framework that unifies these complementary paradigms into a coherent and interpretable architecture. Early modules perform reconstruction-gated filtering and object-level assessment, while higher-level reasoning agents are selectively invoked to interpret semantically ambiguous events. The system employs adaptive escalation thresholds and a publish-subscribe communication backbone, enabling asynchronous coordination and scalable deployment across heterogeneous hardware. Extensive evaluation on large-scale monitoring data demonstrates that the proposed cascade achieves a threefold reduction in latency compared to direct vision-language inference, while maintaining high perceptual fidelity (PSNR = 38.3 dB, SSIM = 0.965) and consistent semantic labeling. The framework advances beyond conventional detection pipelines by combining early-exit efficiency, adaptive multi-agent reasoning, and explainable anomaly attribution, establishing a reproducible and energy-efficient foundation for scalable intelligent visual monitoring.