# PANDA: Towards Generalist Video Anomaly Detection via Agentic AI Engineer

arXiv: https://arxiv.org/abs/2509.26386 · HTML: https://mlsanity.com/paper/2509.26386/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use.
- Builds on 32 of its 49 references that are in the map.
- First posted on arXiv 30 Sep 2025.

## Placed in

- LLM agents and tool use / [Software engineering coding agents](https://mlsanity.com/agents/software-engineering-coding-agents/)

## Builds on (most cited first, 30 of 32)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [Video-LLaVA: Learning United Visual Representation by Alignment Before Projection](https://mlsanity.com/paper/2311.10122/) (2023)
- [Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data](https://mlsanity.com/paper/2107.10833/) (2021)
- [Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation](https://mlsanity.com/paper/2004.09813/) (2020)
- [Real-world Anomaly Detection in Surveillance Videos](https://mlsanity.com/paper/1801.04264/) (2018)
- [Memorizing Normality to Detect Anomaly: Memory-augmented Deep Autoencoder for Unsupervised Anomaly Detection](https://mlsanity.com/paper/1904.02639/) (2019)
- [YOLO-World: Real-Time Open-Vocabulary Object Detection](https://mlsanity.com/paper/2401.17270/) (2024)
- [Future Frame Prediction for Anomaly Detection -- A New Baseline](https://mlsanity.com/paper/1712.09867/) (2017)
- [Learning Temporal Regularity in Video Sequences](https://mlsanity.com/paper/1604.04574/) (2016)
- [Not only Look, but also Listen: Learning Multimodal Violence Detection under Weak Supervision](https://mlsanity.com/paper/2007.04687/) (2020)
- [Weakly-supervised Video Anomaly Detection with Robust Temporal Feature Magnitude Learning](https://mlsanity.com/paper/2101.10030/) (2021)
- [VadCLIP: Adapting Vision-Language Models for Weakly Supervised Video Anomaly Detection](https://mlsanity.com/paper/2308.11681/) (2023)
- [UBnormal: New Benchmark for Supervised Open-Set Video Anomaly Detection](https://mlsanity.com/paper/2111.08644/) (2021)
- [Dual Memory Units with Uncertainty Regulation for Weakly Supervised Video Anomaly Detection](https://mlsanity.com/paper/2302.05160/) (2023)
- [Harnessing Large Language Models for Training-free Video Anomaly Detection](https://mlsanity.com/paper/2404.01014/) (2024)
- [Self-trained Deep Ordinal Regression for End-to-End Video Anomaly Detection](https://mlsanity.com/paper/2003.06780/) (2020)
- [Holmes-VAU: Towards Long-term Video Anomaly Understanding at Any Granularity](https://mlsanity.com/paper/2412.06171/) (2024)
- [Holmes-VAD: Towards Unbiased and Explainable Video Anomaly Detection via Multi-modal LLM](https://mlsanity.com/paper/2406.12235/) (2024)
- [Follow the Rules: Reasoning for Video Anomaly Detection with Large Language Models](https://mlsanity.com/paper/2407.10299/) (2024)
- [Video Event Restoration Based on Keyframes for Video Anomaly Detection](https://mlsanity.com/paper/2304.05112/) (2023)
- [Text Prompt with Normality Guidance for Weakly Supervised Video Anomaly Detection](https://mlsanity.com/paper/2404.08531/) (2024)
- [Hawk: Learning to Understand Open-World Video Anomalies](https://mlsanity.com/paper/2405.16886/) (2024)
- [Uncovering What, Why and How: A Comprehensive Benchmark for Causation Understanding of Video Anomaly](https://mlsanity.com/paper/2405.00181/) (2024)
- [Weakly Supervised Video Anomaly Detection and Localization with Spatio-Temporal Prompts](https://mlsanity.com/paper/2408.05905/) (2024)
- [Self-Distilled Masked Auto-Encoders are Efficient Video Anomaly Detectors](https://mlsanity.com/paper/2306.12041/) (2023)
- [Dynamic Local Aggregation Network with Adaptive Clusterer for Anomaly Detection](https://mlsanity.com/paper/2207.10948/) (2022)
- [Deep Learning for Video Anomaly Detection: A Review](https://mlsanity.com/paper/2409.05383/) (2024)

## Built on by (most cited first, 19 of 19)

- [Learning to Watch: Active Video Anomaly Understanding via Interleaved Policy Optimization](https://mlsanity.com/paper/2607.00622/) (2026)
- [SphereVAD: Training-Free Video Anomaly Detection via Geodesic Inference on the Unit Hypersphere](https://mlsanity.com/paper/2605.08003/) (2026)
- [MAVEN: A Multi-stage Agentic Annotation Pipeline for Video Reasoning Tasks](https://mlsanity.com/paper/2605.21917/) (2026)
- [Rethinking Open-World Video Anomaly Detection: Diagnosing Definition Blindness](https://mlsanity.com/paper/2607.20780/) (2026)
- [AgenticVAU: Multi-Agent Explore-Verify Reasoning for Video Anomaly Understanding](https://mlsanity.com/paper/2608.03779/) (2026)
- [QVAD: A Question-Centric Agentic Framework for Efficient and Training-Free Video Anomaly Detection](https://mlsanity.com/paper/2604.03040/) (2026)
- [ESOM: Efficiently Understanding Streaming Video Anomalies with Open-world Dynamic Definitions](https://mlsanity.com/paper/2604.07772/) (2026)
- [Is Video Anomaly Detection Misframed? Evidence from LLM-Based and Multi-Scene Models](https://mlsanity.com/paper/2605.12725/) (2026)
- [Probe-VAD: Ordinal Likelihood Probing for Training-Free Video Anomaly Detection](https://mlsanity.com/paper/2609.17211/) (2026)
- [VTO: Visual Tool Orchestration for Video Anomaly Detection](https://mlsanity.com/paper/2608.08219/) (2026)
- [Beyond Hazard Resemblance: Contrastive Event Adjudication for Training-Free Video Anomaly Detection](https://mlsanity.com/paper/2608.09908/) (2026)
- [NOVA: Normal-Side Modeling for Training-Free Zero-Shot Video Anomaly Detection](https://mlsanity.com/paper/2609.06360/) (2026)
- [ReactVAU: A Slow-Fast Decoupled Framework for Streaming Video Anomaly Understanding](https://mlsanity.com/paper/2609.07941/) (2026)
- [EvoSherlock: Towards Agentic Lifelong Evolution for Unseen Long-Tailed Security-Critical Events in Videos](https://mlsanity.com/paper/2609.19201/) (2026)
- [Agentic Video Understanding: A Survey](https://mlsanity.com/paper/2609.31713/) (2026)
- [Representation--Behavior Alignment for Explainable Weakly-Supervised Video Anomaly Detection](https://mlsanity.com/paper/2610.05129/) (2026)
- [RoMod: Temporal Routing Modulation via Mixture-of-Experts for Video Anomaly Detection](https://mlsanity.com/paper/2610.05131/) (2026)
- [Learning Temporal Evidence Selection for Video Anomaly Understanding](https://mlsanity.com/paper/2610.16299/) (2026)
- [Anomaly-Guided Observation and Two-Level Semantic Feedback for Training-Free Video Anomaly Analysis](https://mlsanity.com/paper/2610.16324/) (2026)

## Abstract (arXiv)

Video anomaly detection (VAD) is a critical yet challenging task due to the complex and diverse nature of real-world scenarios. Previous methods typically rely on domain-specific training data and manual adjustments when applying to new scenarios and unseen anomaly types, suffering from high labor costs and limited generalization. Therefore, we aim to achieve generalist VAD, ie, automatically handle any scene and any anomaly types without training data or human involvement. In this work, we propose PANDA, an agentic AI engineer based on MLLMs. Specifically, we achieve PANDA by comprehensively devising four key capabilities: (1) self-adaptive scene-aware strategy planning, (2) goal-driven heuristic reasoning, (3) tool-augmented self-reflection, and (4) self-improving chain-of-memory. Concretely, we develop a self-adaptive scene-aware RAG mechanism, enabling PANDA to retrieve anomaly-specific knowledge for anomaly detection strategy planning. Next, we introduce a latent anomaly-guided heuristic prompt strategy to enhance reasoning precision. Furthermore, PANDA employs a progressive reflection mechanism alongside a suite of context-aware tools to iteratively refine decision-making in complex scenarios. Finally, a chain-of-memory mechanism enables PANDA to leverage historical experiences for continual performance improvement. Extensive experiments demonstrate that PANDA achieves state-of-the-art performance in multi-scenario, open-set, and complex scenario settings without training and manual involvement, validating its generalizable and robust anomaly detection capability. Code is released at https://github.com/showlab/PANDA.