# ALARM: Automated MLLM-Based Anomaly Detection in Complex-EnviRonment Monitoring with Uncertainty Quantification

arXiv: https://arxiv.org/abs/2512.03101 · HTML: https://mlsanity.com/paper/2512.03101/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Chart understanding and anomaly detection, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 39 of its 76 references that are in the map.
- First posted on arXiv 1 Dec 2025.

## Placed in

- Multimodal LLMs / vision-language models / [Chart understanding and anomaly detection](https://mlsanity.com/mllm/chart-understanding-and-anomaly-detection/)

## Builds on (most cited first, 30 of 39)

- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations](https://mlsanity.com/paper/2312.06674/) (2023)
- [Selective Classification for Deep Neural Networks](https://mlsanity.com/paper/1705.08500/) (2017)
- [Memorizing Normality to Detect Anomaly: Memory-augmented Deep Autoencoder for Unsupervised Anomaly Detection](https://mlsanity.com/paper/1904.02639/) (2019)
- [Classification with Valid and Adaptive Coverage](https://mlsanity.com/paper/2006.02544/) (2020)
- [Future Frame Prediction for Anomaly Detection -- A New Baseline](https://mlsanity.com/paper/1712.09867/) (2017)
- [SelectiveNet: A Deep Neural Network with an Integrated Reject Option](https://mlsanity.com/paper/1901.09192/) (2019)
- [R-Judge: Benchmarking Safety Risk Awareness for LLM Agents](https://mlsanity.com/paper/2401.10019/) (2024)
- [Least Ambiguous Set-Valued Classifiers with Bounded Error Levels](https://mlsanity.com/paper/1609.00451/) (2016)
- [Learning Temporal Regularity in Video Sequences](https://mlsanity.com/paper/1604.04574/) (2016)
- [Predict Responsibly: Improving Fairness and Accuracy by Learning to Defer](https://mlsanity.com/paper/1711.06664/) (2017)
- [Weakly-supervised Video Anomaly Detection with Robust Temporal Feature Magnitude Learning](https://mlsanity.com/paper/2101.10030/) (2021)
- [Kernel Language Entropy: Fine-grained Uncertainty Quantification for LLMs from Semantic Similarities](https://mlsanity.com/paper/2405.20003/) (2024)
- [Quantifying Uncertainty in Answers from any Language Model and Enhancing their Trustworthiness](https://mlsanity.com/paper/2308.16175/) (2023)
- [Decomposing Uncertainty for Large Language Models through Input Clarification Ensembling](https://mlsanity.com/paper/2311.08718/) (2023)
- [Object-centric Auto-encoders and Dummy Anomalies for Abnormal Event Detection in Video](https://mlsanity.com/paper/1812.04960/) (2018)
- [Graph Embedded Pose Clustering for Anomaly Detection](https://mlsanity.com/paper/1912.11850/) (2019)
- [Learning Normal Dynamics in Videos with Meta Prototype Network](https://mlsanity.com/paper/2104.06689/) (2021)
- [Holmes-VAD: Towards Unbiased and Explainable Video Anomaly Detection via Multi-modal LLM](https://mlsanity.com/paper/2406.12235/) (2024)
- [Optimal strategies for reject option classifiers](https://mlsanity.com/paper/2101.12523/) (2021)
- [Video Anomaly Detection and Explanation via Large Language Models](https://mlsanity.com/paper/2401.05702/) (2024)
- [An overview of deep learning based methods for unsupervised and semi-supervised anomaly detection in videos](https://mlsanity.com/paper/1801.03149/) (2018)
- [Uncertainty Quantification for In-Context Learning of Large Language Models](https://mlsanity.com/paper/2402.10189/) (2024)
- [Do LLMs Understand Visual Anomalies? Uncovering LLM's Capabilities in Zero-shot Anomaly Detection](https://mlsanity.com/paper/2404.09654/) (2024)
- [Position: Uncertainty Quantification Needs Reassessment for Large-language Model Agents](https://mlsanity.com/paper/2505.22655/) (2025)
- [LLM Uncertainty Quantification through Directional Entailment Graph and Claim Level Response Augmentation](https://mlsanity.com/paper/2407.00994/) (2024)
- [ER-Reason: A Benchmark Dataset for LLM Clinical Reasoning in the Emergency Room](https://mlsanity.com/paper/2505.22919/) (2025)
- [Customizing Visual-Language Foundation Models for Multi-modal Anomaly Detection and Reasoning](https://mlsanity.com/paper/2403.11083/) (2024)
- [VANE-Bench: Video Anomaly Evaluation Benchmark for Conversational LMMs](https://mlsanity.com/paper/2406.10326/) (2024)
- [LogiCode: an LLM-Driven Framework for Logical Anomaly Detection](https://mlsanity.com/paper/2406.04687/) (2024)

## Built on by (most cited first, 2 of 2)

- [Team, Then Trim: An Assembly-Line LLM Framework for High-Quality Tabular Data Generation](https://mlsanity.com/paper/2602.04785/) (2026)
- [A Role-Based LLM Framework for Structured Information Extraction from Healthy Food Policies](https://mlsanity.com/paper/2604.01529/) (2026)

## Abstract (arXiv)

The advance of Large Language Models (LLMs) has greatly stimulated research interest in developing multi-modal LLM (MLLM)-based visual anomaly detection (VAD) algorithms that can be deployed in complex environments. The challenge is that in these complex environments, the anomalies are sometimes highly contextual and also ambiguous, and thereby, uncertainty quantification (UQ) is a crucial capacity for an MLLM-based VAD system to succeed. In this paper, we introduce our UQ-supported MLLM-based VAD framework called ALARM. ALARM integrates UQ with quality-assurance techniques like reasoning chain, self-reflection, and MLLM ensemble for robust and accurate performance and is designed based on a rigorous probabilistic inference pipeline and computational process. Extensive empirical evaluations are conducted using the real-world smart-home benchmark data and wound image classification data, which shows ALARM's superior performance and its generic applicability across different domains for reliable decision-making.