# Resource-Efficient Automatic Software Vulnerability Assessment via Knowledge Distillation and Particle Swarm Optimization

arXiv: https://arxiv.org/abs/2508.02840 · HTML: https://mlsanity.com/paper/2508.02840/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Federated, robust and private distillation, one of 12 directions in Knowledge distillation.
- Builds on 32 of its 68 references that are in the map.
- First posted on arXiv 30 Jul 2025.

## Placed in

- Knowledge distillation / [Federated, robust and private distillation](https://mlsanity.com/distill/federated-robust-and-private-distillation/)

## Builds on (most cited first, 30 of 32)

- [Knowledge Distillation: A Survey](https://mlsanity.com/paper/2006.05525/) (2020)
- [MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers](https://mlsanity.com/paper/2002.10957/) (2020)
- [ZeRO: Memory Optimizations Toward Training Trillion Parameter Models](https://mlsanity.com/paper/1910.02054/) (2019)
- [Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learning](https://mlsanity.com/paper/1702.03118/) (2017)
- [Model Pruning Enables Efficient Federated Learning on Edge Devices](https://mlsanity.com/paper/1909.12326/) (2019)
- [An Empirical Evaluation of Using Large Language Models for Automated Unit Test Generation](https://mlsanity.com/paper/2302.06527/) (2023)
- [InferFix: End-to-End Program Repair with LLMs](https://mlsanity.com/paper/2303.07263/) (2023)
- [Compressing Large-Scale Transformer-Based Models: A Case Study on BERT](https://mlsanity.com/paper/2002.11985/) (2020)
- [Knowledge Distillation from Internal Representations](https://mlsanity.com/paper/1910.03723/) (2019)
- [Big Code != Big Vocabulary: Open-Vocabulary Models for Source Code](https://mlsanity.com/paper/2003.07914/) (2020)
- [Knowledge Distillation for Multi-task Learning](https://mlsanity.com/paper/2007.06889/) (2020)
- [VulBERTa: Simplified Source Code Pre-Training for Vulnerability Detection](https://mlsanity.com/paper/2205.12424/) (2022)
- [Large Language Model for Vulnerability Detection: Emerging Results and Future Directions](https://mlsanity.com/paper/2401.15468/) (2024)
- [Applying CodeBERT for Automated Program Repair of Java Simple Bugs](https://mlsanity.com/paper/2103.11626/) (2021)
- [TreeBERT: A Tree-Based Pre-Trained Model for Programming Language](https://mlsanity.com/paper/2105.12485/) (2021)
- [Fast and Memory-Efficient Neural Code Completion](https://mlsanity.com/paper/2004.13651/) (2020)
- [ReGVD: Revisiting Graph Neural Networks for Vulnerability Detection](https://mlsanity.com/paper/2110.07317/) (2021)
- [MegaVul: A C/C++ Vulnerability Dataset with Comprehensive Code Representation](https://mlsanity.com/paper/2406.12415/) (2024)
- [Domain Knowledge Matters: Improving Prompts with Fix Templates for Repairing Python Type Errors](https://mlsanity.com/paper/2306.01394/) (2023)
- [Compressing Pre-trained Models of Code into 3 MB](https://mlsanity.com/paper/2208.07120/) (2022)
- [Greening Large Language Models of Code](https://mlsanity.com/paper/2309.04076/) (2023)
- [A Survey on Data-driven Software Vulnerability Assessment and Prioritization](https://mlsanity.com/paper/2107.08364/) (2021)
- [Automated Software Vulnerability Assessment with Concept Drift](https://mlsanity.com/paper/2103.11316/) (2021)
- [An Exploratory Study on Code Attention in BERT](https://mlsanity.com/paper/2204.10200/) (2022)
- [DeepCVA: Automated Commit-level Vulnerability Assessment with Deep Multi-task Learning](https://mlsanity.com/paper/2108.08041/) (2021)
- [LEOPARD: Identifying Vulnerable Code for Vulnerability Assessment through Program Metrics](https://mlsanity.com/paper/1901.11479/) (2019)
- [Investigating the Efficacy of Large Language Models for Code Clone Detection](https://mlsanity.com/paper/2401.13802/) (2024)
- [Cogni-Net: Cognitive Feature Learning through Deep Visual Perception](https://mlsanity.com/paper/1811.00201/) (2018)
- [Low Precision Floating-point Arithmetic for High Performance FPGA-based CNN Acceleration](https://mlsanity.com/paper/2003.03852/) (2020)
- [On the Use of Fine-grained Vulnerable Code Statements for Software Vulnerability Assessment Models](https://mlsanity.com/paper/2203.08417/) (2022)

## Abstract (arXiv)

The increasing complexity of software systems has led to a surge in cybersecurity vulnerabilities, necessitating efficient and scalable solutions for vulnerability assessment. However, the deployment of large pre-trained models in real-world scenarios is hindered by their substantial computational and storage demands. To address this challenge, we propose a novel resource-efficient framework that integrates knowledge distillation and particle swarm optimization to enable automated vulnerability assessment. Our framework employs a two-stage approach: First, particle swarm optimization is utilized to optimize the architecture of a compact student model, balancing computational efficiency and model capacity. Second, knowledge distillation is applied to transfer critical vulnerability assessment knowledge from a large teacher model to the optimized student model. This process significantly reduces the model size while maintaining high performance. Experimental results on an enhanced MegaVul dataset, comprising 12,071 CVSS (Common Vulnerability Scoring System) v3 annotated vulnerabilities, demonstrate the effectiveness of our approach. Our approach achieves a 99.4% reduction in model size while retaining 89.3% of the original model's accuracy. Furthermore, it outperforms state-of-the-art baselines by 1.7% in accuracy with 60% fewer parameters. The framework also reduces training time by 72.1% and architecture search time by 34.88% compared to traditional genetic algorithms.