mlsanityData to 7 Oct 2026

Modality-Aware Out-of-Distribution Detection for Multi-Modal Action Recognition

arXiv 2606.24404 ↗First posted 23 Jun 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

The incorporation of additional modalities into action recognition models increases their performance across a wide range of settings. However, how this additional information can contribute to making the models more robust remains underexplored, particularly for the case of multi-modal out-of-distribution (OOD) detection. While methods exist that regularize the multi-modal training process with OOD detection in mind, they still apply off-the-shelf OOD detectors designed for the uni-modal case during inference, discarding important information. Based on an interesting relationship we find between the multi-modal and uni-modal predictions, we propose to use this signal to build a post-hoc detector explicitly designed for the multi-modal scenario. We combine this new source of information with a feature-space score, which detects off-manifold samples in the multi-modal space, and normalize them by the multi-modal logits. In doing so, the proposed hybrid detector is compatible with existing training-time approaches and consistently improves performance. Experiments on a wide range of established datasets from the MultiOOD benchmark show that, on average, our approach outperforms the state of the art. Our results show the importance of explicitly considering the different modalities at inference time for multi-modal OOD detection.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 34 of its 43 references that are in the map. First posted on arXiv 23 Jun 2026.

Builds on

34 of 43 references.

References · 34

  1. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
  2. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
  3. 2018SlowFast Networks for Video Recognitioncited by 1,862
  4. 2018A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attackscited by 1,472
  5. 2017Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networkscited by 1,325
  6. 2020Energy-based Out-of-distribution Detectioncited by 1,296
  7. 2018Deep Anomaly Detection with Outlier Exposurecited by 939
  8. 2021Generalized Out-of-Distribution Detection: A Surveycited by 796
  9. 2018Scaling Egocentric Vision: The EPIC-KITCHENS Datasetcited by 789
  10. 2022Out-of-Distribution Detection with Deep Nearest Neighborscited by 596
  11. 2019Scaling Out-of-Distribution Detection for Real-World Settingscited by 465
  12. 2022ViM: Out-Of-Distribution with Virtual-logit Matchingcited by 404
22 more
  1. 2017Training Confidence-calibrated Classifiers for Detecting Out-of-Distribution Samplescited by 389
  2. 2021On the Importance of Gradients for Detecting Distributional Shifts in the Wildcited by 338
  3. 2018A Short Note about Kinetics-600cited by 331
  4. 2022VOS: Learning What You Don't Know by Virtual Outlier Synthesiscited by 300
  5. 2018Learning Confidence for Out-of-Distribution Detection in Neural Networkscited by 245
  6. 2020Contrastive Training for Improved Out-of-Distribution Detectioncited by 186
  7. 2020Human Action Recognition from Various Data Modalities: A Reviewcited by 162
  8. 2023Dream the Impossible: Outlier Imagination with Diffusion Modelscited by 77
  9. 2015P-CNN: Pose-based CNN Features for Action Recognitioncited by 58
  10. 2019Detecting semantic anomaliescited by 50
  11. 2023SimMMDG: A Simple and Effective Framework for Multi-modal Domain Generalizationcited by 48
  12. 2025Mahalanobis++: Improving OOD Detection via Feature Normalizationcited by 30
  13. 2020Why is the Mahalanobis Distance Effective for Anomaly Detection?cited by 28
  14. 2024MultiOOD: Scaling Out-of-Distribution Detection for Multiple Modalitiescited by 28
  15. 2024DPU: Dynamic Prototype Updating for Multimodal Out-of-Distribution Detectioncited by 28
  16. 2023GradOrth: A Simple yet Efficient Out-of-Distribution Detection with Orthogonal Projection of Gradientscited by 21
  17. 2024Gradient-Regularized Out-of-Distribution Detectioncited by 13
  18. 2025CADRef: Robust Out-of-Distribution Detection via Class-Aware Decoupled Relative Feature Leveragingcited by 8
  19. 2025Extremely Simple Multimodal Outlier Synthesis for Out-of-Distribution Detection and Segmentationcited by 8
  20. 2021Data Invariants to Understand Unsupervised Out-of-Distribution Detectioncited by 5
  21. 2024Non-Linear Outlier Synthesis for Out-of-Distribution Detectioncited by 4
  22. 2025Revisiting Logit Distributions for Reliable Out-of-Distribution Detectioncited by 4

Built on by

Not cited yet.