# Modality-Aware Contrastive Instance Learning with Self-Distillation for Weakly-Supervised Audio-Visual Violence Detection

arXiv: https://arxiv.org/abs/2207.05500 · HTML: https://mlsanity.com/paper/2207.05500/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Placed in Speech and audio distillation, one of 12 directions in Knowledge distillation.
- Builds on 49 of its 69 references that are in the map.
- First posted on arXiv 12 Jul 2022.

## Placed in

- Knowledge distillation / [Speech and audio distillation](https://mlsanity.com/distill/speech-and-audio-distillation/)

## Builds on (most cited first, 30 of 49)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination](https://mlsanity.com/paper/1805.01978/) (2018)
- [Big Self-Supervised Models are Strong Semi-Supervised Learners](https://mlsanity.com/paper/2006.10029/) (2020)
- [Self-training with Noisy Student improves ImageNet classification](https://mlsanity.com/paper/1911.04252/) (2019)
- [Contrastive Representation Distillation](https://mlsanity.com/paper/1910.10699/) (2019)
- [CNN Architectures for Large-Scale Audio Classification](https://mlsanity.com/paper/1609.09430/) (2016)
- [Real-world Anomaly Detection in Surveillance Videos](https://mlsanity.com/paper/1801.04264/) (2018)
- [Look, Listen and Learn](https://mlsanity.com/paper/1705.08168/) (2017)
- [Future Frame Prediction for Anomaly Detection -- A New Baseline](https://mlsanity.com/paper/1712.09867/) (2017)
- [Audio-Visual Scene Analysis with Self-Supervised Multisensory Features](https://mlsanity.com/paper/1804.03641/) (2018)
- [Objects that Sound](https://mlsanity.com/paper/1712.06651/) (2017)
- [Learning Temporal Regularity in Video Sequences](https://mlsanity.com/paper/1604.04574/) (2016)
- [Cooperative Learning of Audio and Video Models from Self-Supervised Synchronization](https://mlsanity.com/paper/1807.00230/) (2018)
- [UNIMO: Towards Unified-Modal Understanding and Generation via Cross-Modal Contrastive Learning](https://mlsanity.com/paper/2012.15409/) (2020)
- [Not only Look, but also Listen: Learning Multimodal Violence Detection under Weak Supervision](https://mlsanity.com/paper/2007.04687/) (2020)
- [Weakly-supervised Video Anomaly Detection with Robust Temporal Feature Magnitude Learning](https://mlsanity.com/paper/2101.10030/) (2021)
- [Vision-Language Pre-Training with Triple Contrastive Learning](https://mlsanity.com/paper/2202.10401/) (2022)
- [Audio-Visual Instance Discrimination with Cross-Modal Agreement](https://mlsanity.com/paper/2004.12943/) (2020)
- [Unified Multisensory Perception: Weakly-Supervised Audio-Visual Video Parsing](https://mlsanity.com/paper/2007.10558/) (2020)
- [MIST: Multiple Instance Self-Training Framework for Video Anomaly Detection](https://mlsanity.com/paper/2104.01633/) (2021)

## Built on by (most cited first, 26 of 26)

- [CLIP-TSA: CLIP-Assisted Temporal Self-Attention for Weakly-Supervised Video Anomaly Detection](https://mlsanity.com/paper/2212.05136/) (2022)
- [Generalized Video Anomaly Event Detection: Systematic Taxonomy and Comparison of Deep Models](https://mlsanity.com/paper/2302.05087/) (2023)
- [Learning Prompt-Enhanced Context Features for Weakly-Supervised Video Anomaly Detection](https://mlsanity.com/paper/2306.14451/) (2023)
- [Deep Learning for Video Anomaly Detection: A Review](https://mlsanity.com/paper/2409.05383/) (2024)
- [BatchNorm-based Weakly Supervised Video Anomaly Detection](https://mlsanity.com/paper/2311.15367/) (2023)
- [Networking Systems for Video Anomaly Detection: A Tutorial and Survey](https://mlsanity.com/paper/2405.10347/) (2024)
- [VADTree: Explainable Training-Free Video Anomaly Detection via Hierarchical Granularity-Aware Tree](https://mlsanity.com/paper/2510.22693/) (2025)
- [Learning Weakly Supervised Audio-Visual Violence Detection in Hyperbolic Space](https://mlsanity.com/paper/2305.18797/) (2023)
- [Cross-Modal Fusion and Attention Mechanism for Weakly Supervised Video Anomaly Detection](https://mlsanity.com/paper/2412.20455/) (2024)
- [GlanceVAD: Exploring Glance Supervision for Label-efficient Video Anomaly Detection](https://mlsanity.com/paper/2403.06154/) (2024)
- [AVadCLIP: Audio-Visual Collaboration for Robust Video Anomaly Detection](https://mlsanity.com/paper/2504.04495/) (2025)
- [Vision-Language Models Assisted Unsupervised Video Anomaly Detection](https://mlsanity.com/paper/2409.14109/) (2024)
- [Mixture of Experts Guided by Gaussian Splatters Matters: A new Approach to Weakly-Supervised Video Anomaly Detection](https://mlsanity.com/paper/2508.06318/) (2025)
- [Aligning First, Then Fusing: A Novel Weakly Supervised Multimodal Violence Detection Method](https://mlsanity.com/paper/2501.07496/) (2025)
- [The Evolution of Video Anomaly Detection: A Unified Framework from DNN to MLLM](https://mlsanity.com/paper/2507.21649/) (2025)
- [Multi-scale Bottleneck Transformer for Weakly Supervised Multimodal Violence Detection](https://mlsanity.com/paper/2405.05130/) (2024)
- [Just Dance with π! A Poly-modal Inductor for Weakly-supervised Video Anomaly Detection](https://mlsanity.com/paper/2505.13123/) (2025)
- [Learning Event Completeness for Weakly Supervised Video Anomaly Detection](https://mlsanity.com/paper/2506.13095/) (2025)
- [Weakly Supervised Video Anomaly Detection with Anomaly-Connected Components and Intention Reasoning](https://mlsanity.com/paper/2603.00550/) (2026)
- [Multimodal Attention-Enhanced Feature Fusion-based Weekly Supervised Anomaly Violence Detection](https://mlsanity.com/paper/2409.11223/) (2024)
- [Labeling Comic Mischief Content in Online Videos with a Multimodal Hierarchical-Cross-Attention Model](https://mlsanity.com/paper/2406.07841/) (2024)
- [AVAR-Net: A Lightweight Audio-Visual Anomaly Recognition Framework with a Benchmark Dataset](https://mlsanity.com/paper/2510.13630/) (2025)
- [Enhancing Weakly Supervised Multimodal Video Anomaly Detection through Text Guidance](https://mlsanity.com/paper/2602.10549/) (2026)
- [RobustA: Robust Anomaly Detection in Multimodal Data](https://mlsanity.com/paper/2511.07276/) (2025)
- [AViS-Mamba: Adaptive Visual Steering of Audio State-Space Dynamics for Violence Detection](https://mlsanity.com/paper/2604.03329/) (2026)
- [Look Inside Each Video: Rethinking How Video Anomaly Detection Is Evaluated](https://mlsanity.com/paper/2608.21854/) (2026)

## Abstract (arXiv)

Weakly-supervised audio-visual violence detection aims to distinguish snippets containing multimodal violence events with video-level labels. Many prior works perform audio-visual integration and interaction in an early or intermediate manner, yet overlooking the modality heterogeneousness over the weakly-supervised setting. In this paper, we analyze the modality asynchrony and undifferentiated instances phenomena of the multiple instance learning (MIL) procedure, and further investigate its negative impact on weakly-supervised audio-visual learning. To address these issues, we propose a modality-aware contrastive instance learning with self-distillation (MACIL-SD) strategy. Specifically, we leverage a lightweight two-stream network to generate audio and visual bags, in which unimodal background, violent, and normal instances are clustered into semi-bags in an unsupervised way. Then audio and visual violent semi-bag representations are assembled as positive pairs, and violent semi-bags are combined with background and normal instances in the opposite modality as contrastive negative pairs. Furthermore, a self-distillation module is applied to transfer unimodal visual knowledge to the audio-visual model, which alleviates noises and closes the semantic gap between unimodal and multimodal features. Experiments show that our framework outperforms previous methods with lower complexity on the large-scale XD-Violence dataset. Results also demonstrate that our proposed approach can be used as plug-in modules to enhance other networks. Codes are available at https://github.com/JustinYuu/MACIL_SD.