AST-PAC: AST-guided Membership Inference for Code
On the map: LLM safety and security → Membership inference attacks
Abstract (arXiv)
Code Large Language Models are frequently trained on massive datasets containing restrictively licensed source code. This creates urgent data governance and copyright challenges. Membership Inference Attacks (MIAs) can serve as an auditing mechanism to detect unauthorized data usage in models. While attacks like the Loss Attack provide a baseline, more involved methods like Polarized Augment Calibration (PAC) remain underexplored in the code domain. This paper presents an exploratory study evaluating these methods on 3B--7B parameter code models. We find that while PAC generally outperforms the Loss baseline, its effectiveness relies on augmentation strategies that disregard the rigid syntax of code, leading to performance degradation on larger, complex files. To address this, we introduce AST-PAC, a domain-specific adaptation that utilizes Abstract Syntax Tree (AST) based perturbations to generate syntactically valid calibration samples. Preliminary results indicate that AST-PAC improves as syntactic size grows, where PAC degrades, but under-mutates small files and underperforms on alphanumeric-rich code. Overall, the findings motivate future work on syntax-aware and size-adaptive calibration as a prerequisite for reliable provenance auditing of code language models.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Membership inference attacks, one of 12 directions in LLM safety and security. Builds on 16 of its 25 references that are in the map. First posted on arXiv 30 Jan 2026.
Builds on
16 of 25 references.
Closest ancestors · 7
Same direction, newest first
- 2025Membership Inference Attacks on Large-Scale Models: A Survey25 Mar
- 2025How Much Do Code Language Models Remember? An Investigation on Data Extraction Attacks before and after Fine-tuning29 Jan
- 2024SoK: Membership Inference Attacks on LLMs are Rushing Nowhere (and How to Fix It)25 Jun
- 2024Does Your Neural Code Completion Model Use My Code? A Membership Inference Approach22 Apr
- 2023Code Membership Inference for Detecting Unauthorized Data Use in Code Pre-trained Language Models12 Dec
- 2021Membership Inference Attacks From First Principles7 Dec
- 2021Membership Inference Attacks on Machine Learning: A Survey14 Mar
Other references · 9
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
- 2020Extracting Training Data from Large Language Modelscited by 2,144
- 2024StarCoder 2 and The Stack v2: The Next Generationcited by 484
- 2024Robustness, Security, Privacy, Explainability, Efficiency, and Usability of Large Language Models for Codecited by 26
- 2024Data Contamination Calibration for Black-box LLMscited by 23
- 2023The (ab)use of Open Source Code to Train Large Language Modelscited by 9
- 2024An Exploratory Investigation into Code License Infringements in Large Language Model Training Datasetscited by 8
- 2025Mellum: Production-Grade in-IDE Contextual Code Completion with Multi-File Project Understandingcited by 2
Built on by
Not cited yet.