Frugal Federated Learning for Violence Detection: A Comparison of LoRA-Tuned VLMs and Personalized CNNs
On the map: Multimodal LLMs / vision-language models → Prompt learning and adaptation of CLIP
Abstract (arXiv)
We examine frugal federated learning approaches to violence detection by comparing two complementary strategies: (i) zero-shot and federated fine-tuning of vision-language models (VLMs), and (ii) personalized training of a compact 3D convolutional neural network (CNN3D). Using LLaVA-7B and a 65.8M parameter CNN3D as representative cases, we evaluate accuracy, calibration, and energy usage under realistic non-IID settings. Both approaches exceed 90% accuracy. CNN3D slightly outperforms Low-Rank Adaptation(LoRA)-tuned VLMs in ROC AUC and log loss, while using less energy. VLMs remain favorable for contextual reasoning and multimodal inference. We quantify energy and CO2 emissions across training and inference, and analyze sustainability trade-offs for deployment. To our knowledge, this is the first comparative study of LoRA-tuned vision-language models and personalized CNNs for federated violence detection, with an emphasis on energy efficiency and environmental metrics. These findings support a hybrid model: lightweight CNNs for routine classification, with selective VLM activation for complex or descriptive scenarios. The resulting framework offers a reproducible baseline for responsible, resource-aware AI in video surveillance, with extensions toward real-time, multimodal, and lifecycle-aware systems.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Prompt learning and adaptation of CLIP, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 13 of its 23 references that are in the map. First posted on arXiv 20 Oct 2025.
Builds on
13 of 23 references.
Closest ancestors · 1
Same direction, newest first
Other references · 12
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
- 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
- 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
- 2019Quantifying the Carbon Emissions of Machine Learningcited by 521
- 2020Towards the Systematic Reporting of the Energy and Carbon Footprints of Machine Learningcited by 249
- 2020Carbontracker: Tracking and Predicting the Carbon Footprint of Training Deep Learning Modelscited by 169
- 2024FDLoRA: Personalized Federated Learning of Large Language Model via Dual LoRA Tuningcited by 59
- 2019RWF-2000: An Open Large Scale Video Database for Violence Detectioncited by 39
- 2024LoRA-FAIR: Federated LoRA Fine-Tuning with Aggregation and Initialization Refinementcited by 38
- 2024FLoCoRA: Federated learning compression with low-rank adaptationcited by 3
- 2023Balancing Accuracy and Training Time in Federated Learning for Violence Detection in Surveillance Videos: A Study of Neural Network Architecturescited by 2
- 2025Exploring Personalized Federated Learning Architectures for Violence Detection in Surveillance Videoscited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Federated Learning for Video Violence Detection: Complementary Roles of Lightweight CNNs and Vision-Language Models for Energy-Efficient Use
- 2025FedVLM: Scalable Personalized Vision-Language Models through Federated Learning
- 2025FedVLMBench: Benchmarking Federated Fine-Tuning of Vision-Language Models
- 2026FedSSMCoOp: SSM Encoders for light-weight Federated Prompt Learning for Few-shot Classification
- 2026Federated Binary Gating with Server-Side Vision-Language Inference for Surveillance Anomaly Classification
- 2026UniFed-VLM: Federated Instruction Tuning for Vision-Language Models with Multiple Heterogeneity
- 2026Parser-Free VLM Verification for Federated Weakly Supervised Video Anomaly Detection
- 2026Decoupled Training with Local Reinforcement Fine-Tuning in Federated Learning