Frequency-Aware Continual Learning for Smart Contract Vulnerability Detection with Large Language Models
Not on the map. Why?
Abstract (arXiv)
Smart contract vulnerability detection with Large Language Models (LLMs) faces three causally linked challenges. First, new vulnerability categories demand parameter-efficient adaptation, since full retraining is prohibitive for sequentially arriving tasks. Second, training per-task adapters on a shared backbone causes catastrophic forgetting of previously learned vulnerabilities. Third, the resulting multiplicity of adapters must be consolidated into a single model, since task identity is unknown at inference time. Each challenge arises directly from the solution to its predecessor, making an integrated framework essential. We propose a three-stage pipeline in which each stage addresses one challenge and feeds into the next. The adaptation stage uses Frequency-Aware Low-Rank Adaptation (FA-LoRA), which performs adaptation in the Fourier domain with per-frequency importance gates, requiring only 0.4% trainable parameters while outperforming standard LoRA and QLoRA. The continual learning stage applies Forget-Aware Replay (FAR), which uses these frequency gates to estimate per-sample forgetting risk via loss dynamics and prioritizes vulnerable knowledge for rehearsal, achieving an average Micro-F1 of 0.8022 across sequential tasks. The deployment stage employs Anchor-Protected Progressive Merging (APPM), which exploits the asymmetric generalization produced by FAR training to identify the strongest-generalizing adapter as an anchor and consolidates all adapters into a single model via anchor-protected weighted merging with frequency-domain gate competition. APPM achieves a Micro-F1 of 0.8085, within 2.7% of the independent per-task upper bound, at a merge cost of 156 ms and no additional runtime memory. Experiments on DIVE confirm the framework effectively addresses all three challenges for evolving blockchain ecosystems.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 35 of its 46 references that are in the map. First posted on arXiv 20 Aug 2026.
Builds on
35 of 46 references.
References · 35
- 2024The Llama 3 Herd of Modelscited by 14,192
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
- 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
- 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
- 2017Gradient Episodic Memory for Continual Learningcited by 1,871
- 2017Memory Aware Synapses: Learning what (not) to forgetcited by 1,094
- 2022Editing Models with Task Arithmeticcited by 1,076
- 2020Dark Experience for General Continual Learning: a Strong, Simple Baselinecited by 895
- 2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 860
- 2023TIES-Merging: Resolving Interference When Merging Modelscited by 795
- 2023Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunchcited by 679
- 2021Merging Models with Fisher-Weighted Averagingcited by 613
23 more
- 2018Progress & Compress: A scalable framework for continual learningcited by 516
- 2022Dataless Knowledge Fusion by Merging Weights of Language Modelscited by 384
- 2024Model Merging in LLMs, MLLMs, and Beyond: Methods, Theories, Applications and Opportunitiescited by 247
- 2024Continual Learning of Large Language Models: A Comprehensive Surveycited by 228
- 2023Sparse Low-rank Adaptation of Pre-trained Language Modelscited by 96
- 2019Slither: A Static Analysis Framework For Smart Contractscited by 88
- 2024Parameter-Efficient Fine-Tuning with Discrete Fourier Transformcited by 67
- 2023GPTScan: Detecting Logic Vulnerabilities in Smart Contracts by Combining GPT with Program Analysiscited by 65
- 2018Securify: Practical Security Analysis of Smart Contractscited by 61
- 2023When ChatGPT Meets Smart Contract Vulnerability Detection: How Far Are We?cited by 42
- 2024Combining Fine-Tuning and LLM-based Agents for Intuitive Smart Contract Auditing with Justificationscited by 41
- 2024PropertyGPT: LLM-driven Formal Verification of Smart Contracts through Retrieval-Augmented Property Generationcited by 36
- 2018ContractFuzzer: Fuzzing Smart Contracts for Vulnerability Detectioncited by 32
- 2024Sparse High Rank Adapterscited by 17
- 2024FouRA: Fourier Low Rank Adaptationcited by 15
- 2025Cached Multi-Lora Composition for Multi-Concept Image Generationcited by 13
- 2023The Age of Generative AI and AI-Generated Everythingcited by 7
- 2025Smart-LLaMA-DPO: Reinforced Large Language Model for Explainable Smart Contract Vulnerability Detectioncited by 7
- 2025Augmenting Smart Contract Decompiler Output through Fine-grained Dependency Analysis and LLM-facilitated Semantic Recoverycited by 2
- 2024SmartAxe: Detecting Cross-Chain Vulnerabilities in Bridge Smart Contracts via Fine-Grained Static Analysiscited by 1
- 2025WaRA: Wavelet Low-Rank Adaptation for Medical Image Classificationcited by 1
- 2025Satellite: Detecting and Analyzing Smart Contract Vulnerabilities caused by Subcontract Misusecited by 1
- 2025ParaVul: A Parallel Large Language Model and Retrieval-Augmented Framework for Smart Contract Vulnerability Detectioncited by 1
Built on by
Not cited yet.