# DPU: Dynamic Prototype Updating for Multimodal Out-of-Distribution Detection

arXiv: https://arxiv.org/abs/2411.08227 · HTML: https://mlsanity.com/paper/2411.08227/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 52 references that are in the map.
- First posted on arXiv 12 Nov 2024.

## Builds on (most cited first, 7 of 7)

- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [The Kinetics Human Action Video Dataset](https://mlsanity.com/paper/1705.06950/) (2017)
- [Extremely Simple Activation Shaping for Out-of-Distribution Detection](https://mlsanity.com/paper/2209.09858/) (2022)
- [SuperFusion: Multilevel LiDAR-Camera Fusion for Long-Range HD Map Generation](https://mlsanity.com/paper/2211.15656/) (2022)
- [An Empirical Study of Invariant Risk Minimization](https://mlsanity.com/paper/2004.05007/) (2020)
- [MetaOOD: Automatic Selection of OOD Detection Models](https://mlsanity.com/paper/2410.03074/) (2024)
- [LEGO-Learn: Label-Efficient Graph Open-Set Learning](https://mlsanity.com/paper/2410.16386/) (2024)

## Built on by (most cited first, 29 of 29)

- [DecAlign: Hierarchical Cross-Modal Alignment for Decoupled Multimodal Representation Learning](https://mlsanity.com/paper/2503.11892/) (2025)
- [Advances in Multimodal Adaptation and Generalization: From Traditional Approaches to Foundation Models](https://mlsanity.com/paper/2501.18592/) (2025)
- [Secure On-Device Video OOD Detection Without Backpropagation](https://mlsanity.com/paper/2503.06166/) (2025)
- [FORTIS: Benchmarking Over-Privilege in Agent Skills](https://mlsanity.com/paper/2605.09163/) (2026)
- [Towards Robust Multimodal Open-set Test-time Adaptation via Adaptive Entropy-aware Optimization](https://mlsanity.com/paper/2501.13924/) (2025)
- [Charts Are Not Images: On the Challenges of Scientific Chart Editing](https://mlsanity.com/paper/2512.00752/) (2025)
- [JailDAM: Jailbreak Detection with Adaptive Memory for Vision-Language Model](https://mlsanity.com/paper/2504.03770/) (2025)
- [Counterfactual Trace Auditing of LLM Agent Skills](https://mlsanity.com/paper/2605.11946/) (2026)
- [Extremely Simple Multimodal Outlier Synthesis for Out-of-Distribution Detection and Segmentation](https://mlsanity.com/paper/2505.16985/) (2025)
- [Generative AI for Autonomous Driving: Frontiers and Opportunities](https://mlsanity.com/paper/2505.08854/) (2025)
- [Graph Synthetic Out-of-Distribution Exposure with Large Language Models](https://mlsanity.com/paper/2504.21198/) (2025)
- [Multi-Cache Enhanced Prototype Learning for Test-Time Generalization of Vision-Language Models](https://mlsanity.com/paper/2508.01225/) (2025)
- [RaMem: Contextual Reinstatement for Long-term Agentic Memory](https://mlsanity.com/paper/2606.22844/) (2026)
- [MambaTAD: When State-Space Models Meet Long-Range Temporal Action Detection](https://mlsanity.com/paper/2511.17929/) (2025)
- [To Trust Or Not To Trust Your Vision-Language Model's Prediction](https://mlsanity.com/paper/2505.23745/) (2025)
- [Calibrated Multimodal Representation Learning with Missing Modalities](https://mlsanity.com/paper/2511.12034/) (2025)
- [Beyond the Embedding Bottleneck: Adaptive Retrieval-Augmented 3D CT Report Generation](https://mlsanity.com/paper/2603.15822/) (2026)
- [Memory Retrieval for Changing Preferences](https://mlsanity.com/paper/2606.02976/) (2026)
- [Inside the Unfair Judge: A Mechanistic Interpretability Account of LLM-as-Judge Bias](https://mlsanity.com/paper/2607.11871/) (2026)
- [Geometry over Density: Few-Shot Cross-Domain OOD Detection](https://mlsanity.com/paper/2605.03410/) (2026)
- [Adaptive Confidence Regularization for Multimodal Failure Detection](https://mlsanity.com/paper/2603.02200/) (2026)
- [Enhance-then-Balance Modality Collaboration for Robust Multimodal Sentiment Analysis](https://mlsanity.com/paper/2604.12518/) (2026)
- [JigShape: Evaluating Visual-Geometric Reasoning in VLMs through Jigsaw Puzzles](https://mlsanity.com/paper/2607.27670/) (2026)
- [DPL: Decoupled Prototype Learning for Enhancing Robustness of Vision-Language Transformers to Missing Modalities](https://mlsanity.com/paper/2505.08283/) (2025)
- [How to Achieve Prototypical Birth and Death for OOD Detection?](https://mlsanity.com/paper/2603.15650/) (2026)
- [Are We Making Progress in Multimodal Domain Generalization? A Comprehensive Benchmark Study](https://mlsanity.com/paper/2605.06643/) (2026)
- [Modality-Aware Out-of-Distribution Detection for Multi-Modal Action Recognition](https://mlsanity.com/paper/2606.24404/) (2026)
- [Multi-Modal Anomaly Detection: A Survey](https://mlsanity.com/paper/2608.24937/) (2026)
- [Controlled Decoding Attacks on Black-Box LLMs](https://mlsanity.com/paper/2609.36956/) (2026)

## Abstract (arXiv)

Out-of-distribution (OOD) detection is essential for ensuring the robustness of machine learning models by identifying samples that deviate from the training distribution. While traditional OOD detection has primarily focused on single-modality inputs, such as images, recent advances in multimodal models have demonstrated the potential of leveraging multiple modalities (e.g., video, optical flow, audio) to enhance detection performance. However, existing methods often overlook intra-class variability within in-distribution (ID) data, assuming that samples of the same class are perfectly cohesive and consistent. This assumption can lead to performance degradation, especially when prediction discrepancies are uniformly amplified across all samples. To address this issue, we propose Dynamic Prototype Updating (DPU), a novel plug-and-play framework for multimodal OOD detection that accounts for intra-class variations. Our method dynamically updates class center representations for each class by measuring the variance of similar samples within each batch, enabling adaptive adjustments. This approach allows us to amplify prediction discrepancies based on the updated class centers, thereby improving the model's robustness and generalization across different modalities. Extensive experiments on two tasks, five datasets, and nine base OOD algorithms demonstrate that DPU significantly improves OOD detection performance, setting a new state-of-the-art in multimodal OOD detection, with improvements of up to 80 percent in Far-OOD detection. To facilitate accessibility and reproducibility, our code is publicly available on GitHub.