# How Do Semantically Equivalent Code Transformations Impact Membership Inference on LLMs for Code?

arXiv: https://arxiv.org/abs/2512.15468 · HTML: https://mlsanity.com/paper/2512.15468/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Membership inference attacks, one of 12 directions in LLM safety and security.
- Builds on 31 of its 55 references that are in the map.
- First posted on arXiv 17 Dec 2025.

## Placed in

- LLM safety and security / [Membership inference attacks](https://mlsanity.com/safety/membership-inference-attacks/)

## Builds on (most cited first, 30 of 31)

- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [Code Llama: Open Foundation Models for Code](https://mlsanity.com/paper/2308.12950/) (2023)
- [DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligence](https://mlsanity.com/paper/2401.14196/) (2024)
- [CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis](https://mlsanity.com/paper/2203.13474/) (2022)
- [StarCoder: may the source be with you!](https://mlsanity.com/paper/2305.06161/) (2023)
- [CodeSearchNet Challenge: Evaluating the State of Semantic Code Search](https://mlsanity.com/paper/1909.09436/) (2019)
- [Differentially Private Fine-tuning of Language Models](https://mlsanity.com/paper/2110.06500/) (2021)
- [The Stack: 3 TB of permissively licensed source code](https://mlsanity.com/paper/2211.15533/) (2022)
- [Membership Inference Attacks against Language Models via Neighbourhood Comparison](https://mlsanity.com/paper/2305.18462/) (2023)
- [Machine Learning with Membership Privacy using Adversarial Regularization](https://mlsanity.com/paper/1807.05852/) (2018)
- [Practical Membership Inference Attacks against Fine-tuned Large Language Models via Self-prompt Calibration](https://mlsanity.com/paper/2311.06062/) (2023)
- [DoWhy: An End-to-End Library for Causal Inference](https://mlsanity.com/paper/2011.04216/) (2020)
- [Mitigating Membership Inference Attacks by Self-Distillation Through a Novel Ensemble Architecture](https://mlsanity.com/paper/2110.08324/) (2021)
- [Big Code != Big Vocabulary: Open-Vocabulary Models for Source Code](https://mlsanity.com/paper/2003.07914/) (2020)
- [Stealthy Backdoor Attack for Code Models](https://mlsanity.com/paper/2301.02496/) (2023)
- [On the Generalizability of Neural Program Models with respect to Semantic-Preserving Program Transformations](https://mlsanity.com/paper/2008.01566/) (2020)
- [Unveiling Memorization in Code Models](https://mlsanity.com/paper/2308.09932/) (2023)
- [Assessing the Security of GitHub Copilot Generated Code -- A Targeted Replication Study](https://mlsanity.com/paper/2311.11177/) (2023)
- [Challenging Machine Learning-based Clone Detectors via Semantic-preserving Code Transformations](https://mlsanity.com/paper/2111.10793/) (2021)
- [Gotcha! This Model Uses My Code! Evaluating Membership Leakage Risks in Code Models](https://mlsanity.com/paper/2310.01166/) (2023)
- [Codex Hacks HackerRank: Memorization Issues and a Framework for Code Synthesis Evaluation](https://mlsanity.com/paper/2212.02684/) (2022)
- [LiCoEval: Evaluating LLMs on License Compliance in Code Generation](https://mlsanity.com/paper/2408.02487/) (2024)
- [Evaluating and Explaining Large Language Models for Code Using Syntactic Structures](https://mlsanity.com/paper/2308.03873/) (2023)
- [An Exploratory Investigation into Code License Infringements in Large Language Model Training Datasets](https://mlsanity.com/paper/2403.15230/) (2024)
- [Trained Without My Consent: Detecting Code Inclusion In Language Models Trained on Code](https://mlsanity.com/paper/2402.09299/) (2024)
- [Are Large Language Models Robust in Understanding Code Against Semantics-Preserving Mutations?](https://mlsanity.com/paper/2505.10443/) (2025)
- [Which Syntactic Capabilities Are Statistically Learned by Masked Language Models for Code?](https://mlsanity.com/paper/2401.01512/) (2024)
- [Towards Reliable Evaluation of Neural Program Repair with Natural Robustness Testing](https://mlsanity.com/paper/2402.11892/) (2024)
- [How Quantization Impacts Privacy Risk on LLMs for Code?](https://mlsanity.com/paper/2508.00128/) (2025)
- [Can Identifier Splitting Improve Open-Vocabulary Language Model of Code?](https://mlsanity.com/paper/2201.01988/) (2022)

## Abstract (arXiv)

The success of large language models for code relies on vast amounts of code data, including public open-source repositories, such as GitHub, and private, confidential code from companies. This raises concerns about intellectual property compliance and the potential unauthorized use of license-restricted code. While membership inference (MI) techniques have been proposed to detect such unauthorized usage, their effectiveness can be undermined by semantically equivalent code transformation techniques, which modify code syntax while preserving semantic. In this work, we systematically investigate whether semantically equivalent code transformation rules might be leveraged to evade MI detection. The results reveal that model accuracy drops by only 1.5% in the worst case for each rule, demonstrating that transformed datasets can effectively serve as substitutes for fine-tuning. Additionally, we find that one of the rules (RenameVariable) reduces MI success by 10.19%, highlighting its potential to obscure the presence of restricted code. To validate these findings, we conduct a causal analysis confirming that variable renaming has the strongest causal effect in disrupting MI detection. Notably, we find that combining multiple transformations does not further reduce MI effectiveness. Our results expose a critical loophole in license compliance enforcement for training large language models for code, showing that MI detection can be substantially weakened by transformation-based obfuscation techniques.