EvoSherlock: Towards Agentic Lifelong Evolution for Unseen Long-Tailed Security-Critical Events in Videos
On the map: LLM agents and tool use → Multimodal and embodied agents
Abstract (arXiv)
Existing Security-oriented Video Understanding (SVU) systems assume a closed world, ie static category sets, abundant labels, and the premise that all event types are known upfront. Real-world security-critical events break these assumptions: they follow long-tailed distributions, new types emerge continuously, and critical security events may offer only a few samples. We formalize this gap as textbfLifelong Evolving Task for Long-Tailed Security-Critical Events in Videos (boldmathL2-SCE), a new task that requires VLMs to continually classify and temporally localize newly emerging security-critical events from scarce samples without forgetting previously learned events. Furthermore, task reveals two critical challenges: (1)~Intra-Event Scarcity, where extreme data scarcity may weaken both classification and temporal localization for new events, and (2)~Inter-Event Interference, where cross-event feature entanglement and representation drift may strengthen catastrophic forgetting. On this basis, we propose method, a causal-enhanced approach orchestrated end-to-end by an Agentic Controller with self-reflective closed-loop control, which includes two core modules: the Intra-Event Causal Video Generation module (CVG) and the Inter-Event Causal Decoupling and Alignment module (CDA), to address the above two challenges, respectively. Especially, this paper constructs a task dataset to simulate real-world incremental conditions. Extensive experiments on our benchmark demonstrate the advantages of method over several advanced baselines. These justify the importance of the proposed task and the effectiveness of method in classifying and temporally localizing emerging security-critical events from scarce samples.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 40 of its 64 references that are in the map. First posted on arXiv 16 Sep 2026.
Builds on
40 of 64 references.
References · 40
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2015Distilling the Knowledge in a Neural Networkcited by 10,076
- 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
- 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,536
- 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
- 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
- 2016Learning without Forgettingcited by 2,465
- 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,313
- 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
- 2017Gradient Episodic Memory for Continual Learningcited by 1,871
- 2019Invariant Risk Minimizationcited by 1,832
- 2020Dark Experience for General Continual Learning: a Strong, Simple Baselinecited by 895
28 more
- 2024WorldSimBench: Towards Video Generation Models as World Simulatorscited by 867
- 2019Large Scale Incremental Learningcited by 727
- 2024Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Surveycited by 641
- 2018Real-world Anomaly Detection in Surveillance Videoscited by 570
- 2020Unbiased Scene Graph Generation from Biased Trainingcited by 355
- 2022ActionFormer: Localizing Moments of Actions with Transformerscited by 308
- 2021Weakly-supervised Video Anomaly Detection with Robust Temporal Feature Magnitude Learningcited by 226
- 2019Learning to Remember: A Synaptic Plasticity Driven Framework for Continual Learningcited by 181
- 2020Continual Prototype Evolution: Learning Online from Non-Stationary Data Streamscited by 143
- 2023TriDet: Temporal Action Detection with Relative Boundary Modelingcited by 141
- 2021Causal Attention for Vision-Language Taskscited by 111
- 2021Counterfactual Zero-Shot and Open-Set Visual Recognitioncited by 95
- 2022Few-Shot Class-Incremental Learning from an Open-Set Perspectivecited by 83
- 2026OpenClaw-RL: Train Any Agent Simply by Talkingcited by 83
- 2025Lifelong Learning of Large Language Model based Agents: A Roadmapcited by 78
- 2024Holmes-VAD: Towards Unbiased and Explainable Video Anomaly Detection via Multi-modal LLMcited by 63
- 2023Generalized Video Anomaly Event Detection: Systematic Taxonomy and Comparison of Deep Modelscited by 59
- 2026Memento-Skills: Let Agents Design Agentscited by 56
- 2023Towards Surveillance Video-and-Language Understanding: New Dataset, Baselines, and Challengescited by 42
- 2024Deep Learning for Video Anomaly Detection: A Reviewcited by 27
- 2025SurveillanceVQA-589K: A Benchmark for Comprehensive Surveillance Video-Language Understanding with Large Modelscited by 18
- 2025Conan: Progressive Learning to Reason Like a Detective over Multi-Scale Visual Evidencecited by 17
- 2025PANDA: Towards Generalist Video Anomaly Detection via Agentic AI Engineercited by 15
- 2025Sherlock: Towards Multi-scene Video Abnormal Event Extraction and Localization via a Global-local Spatial-sensitive LLMcited by 10
- 2024ViLCo-Bench: VIdeo Language COntinual learning Benchmarkcited by 9
- 2024Exploring What Why and How: A Multifaceted Benchmark for Causation Understanding of Video Anomalycited by 9
- 2025Towards Closed-Loop Embodied Empathy Evolution: Probing LLM-Centric Lifelong Empathic Motion Generation in Unseen Scenarioscited by 2
- 2026DeepSVU: Towards In-depth Security-oriented Video Understanding via Unified Physical-world Regularized MoEcited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025VAU-R1: Advancing Video Anomaly Understanding via Reinforcement Fine-Tuning
- 2026Teaching Video Generators to Remember: Eliciting Dynamic Memory for Out-of-Sight State Evolution
- 2026Agentic Video Understanding: A Survey
- 2026Learning to Watch: Active Video Anomaly Understanding via Interleaved Policy Optimization
- 2026CoEvoWhen: Policy-Tool Coevolution for Ultra-Long Video Temporal Grounding
- 2025Live-E2T: Real-time Threat Monitoring in Video via Deduplicated Event Reasoning and Chain-of-Thought
- 2026AgenticVAU: Multi-Agent Explore-Verify Reasoning for Video Anomaly Understanding
- 2026The Low Frequency Trap: Video Language Models Fail at Simple Event Bookkeeping