Reinforcement Learning-based Mixture of Vision Transformers for Video Violence Recognition
On the map: Mixture-of-experts → Vision mixture-of-experts
Abstract (arXiv)
Video violence recognition based on deep learning concerns accurate yet scalable human violence recognition. Currently, most state-of-the-art video violence recognition studies use CNN-based models to represent and categorize videos. However, recent studies suggest that pre-trained transformers are more accurate than CNN-based models on various video analysis benchmarks. Yet these models are not thoroughly evaluated for video violence recognition. This paper introduces a novel transformer-based Mixture of Experts (MoE) video violence recognition system. Through an intelligent combination of large vision transformers and efficient transformer architectures, the proposed system not only takes advantage of the vision transformer architecture but also reduces the cost of utilizing large vision transformers. The proposed architecture maximizes violence recognition system accuracy while actively reducing computational costs through a reinforcement learning-based router. The empirical results show the proposed MoE architecture's superiority over CNN-based models by achieving 92.4% accuracy on the RWF dataset.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Vision mixture-of-experts, one of 12 directions in Mixture-of-experts. Builds on 41 of its 69 references that are in the map. First posted on arXiv 4 Oct 2023.
Builds on
41 of 69 references.
Closest ancestors · 3
Same direction, newest first
- 2023Scaling Vision-Language Models with Sparse Mixture of Experts13 Mar
- 2022RoME: Role-aware Mixture-of-Expert Transformer for Text-to-Video Retrieval26 Jun
- 2022Residual Mixture of Experts20 Apr
Other references · 38
- 2017Attention Is All You Needcited by 60,361
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
- 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
- 2017Non-local Neural Networkscited by 2,293
- 2018SlowFast Networks for Video Recognitioncited by 1,867
- 2021Is Space-Time Attention All You Need for Video Understanding?cited by 1,620
- 2021ViViT: A Video Vision Transformercited by 1,620
- 2022VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Trainingcited by 1,322
- 2021Transformers in Vision: A Surveycited by 1,013
- 2021Scaling Vision Transformerscited by 999
- 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 897
26 more
- 2022Mixture-of-Experts with Expert Choice Routingcited by 623
- 2019Stand-Alone Self-Attention in Vision Modelscited by 542
- 2019Learning agile and dynamic motor skills for legged robotscited by 504
- 2021When Vision Transformers Outperform ResNets without Pre-training or Strong Data Augmentationscited by 260
- 2021Revisiting Skeleton-based Action Recognitioncited by 257
- 2018Human Action Recognition and Prediction: A Surveycited by 227
- 2021Scaling Local Self-Attention for Parameter Efficient Visual Backbonescited by 220
- 2022EdgeViTs: Competing Light-weight CNNs on Mobile Devices with Vision Transformerscited by 116
- 2021Are Convolutional Neural Networks or Transformers more like human vision?cited by 109
- 2021Dueling RL: Reinforcement Learning with Trajectory Preferencescited by 106
- 2020Improving Generalization in Reinforcement Learning with Mixture Regularizationcited by 81
- 2021On the Integration of Self-Attention and Convolutioncited by 74
- 2019Learning Spatio-Temporal Representation with Local and Global Diffusioncited by 61
- 2021Deep Reinforcement Learning in Computer Vision: A Comprehensive Surveycited by 43
- 2017On the Integration of Optical Flow and Action Recognitioncited by 42
- 2019RWF-2000: An Open Large Scale Video Database for Violence Detectioncited by 39
- 2020Dynamic Sampling Networks for Efficient Action Recognition in Videoscited by 34
- 2021Revitalizing CNN Attentions via Transformers in Self-Supervised Visual Representation Learningcited by 34
- 2023Unified Keypoint-based Action Recognition Framework via Structured Keypoint Poolingcited by 19
- 2021Efficient Two-Stream Network for Violence Detection Using Separable Convolutional LSTMcited by 12
- 2021Comparing object recognition in humans and deep convolutional neural networks -- An eye tracking studycited by 12
- 2022SERE: Exploring Feature Self-relation for Self-supervised Transformercited by 12
- 2022Video Violence Recognition and Localization Using a Semi-Supervised Hard Attention Modelcited by 6
- 2023Adaptive Feature Fusion: Enhancing Generalization in Deep Learning Modelscited by 5
- 2022An Overview of Violence Detection Techniques: Current Challenges and Future Directionscited by 3
- 2020A Computation-Efficient CNN System for High-Quality Brain Tumor Segmentationcited by 2
Built on by
Citing papers per year: 2025: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2023Distilling Knowledge from CNN-Transformer Models for Enhanced Human Action Recognition
- 2025Federated Learning for Video Violence Detection: Complementary Roles of Lightweight CNNs and Vision-Language Models for Energy-Efficient Use
- 2025FuseMamba-VD: Dual Branch VideoMamba with Gated Class Token Fusion for Violence Detection
- 2026UNIVID: Unified Vision-Language Model for Video Moderation
- 2025Advancing Vision-based Human Action Recognition: Exploring Vision-Language CLIP Model for Generalisation in Domain-Independent Tasks
- 2023Multi-view knowledge distillation transformer for human action recognition
- 2025Frugal Federated Learning for Violence Detection: A Comparison of LoRA-Tuned VLMs and Personalized CNNs
- 2025Robustness Evaluation for Video Models with Reinforcement Learning