# SERSEM: Selective Entropy-Weighted Scoring for Membership Inference in Code Language Models

arXiv: https://arxiv.org/abs/2604.01147 · HTML: https://mlsanity.com/paper/2604.01147/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Membership inference attacks, one of 12 directions in LLM safety and security.
- Builds on 7 of its 14 references that are in the map.
- First posted on arXiv 1 Apr 2026.

## Placed in

- LLM safety and security / [Membership inference attacks](https://mlsanity.com/safety/membership-inference-attacks/)

## Builds on (most cited first, 7 of 7)

- [StarCoder 2 and The Stack v2: The Next Generation](https://mlsanity.com/paper/2402.19173/) (2024)
- [The Stack: 3 TB of permissively licensed source code](https://mlsanity.com/paper/2211.15533/) (2022)
- [Traces of Memorisation in Large Language Models for Code](https://mlsanity.com/paper/2312.11658/) (2023)
- [Adaptive Pre-training Data Detection for Large Language Models via Surprising Tokens](https://mlsanity.com/paper/2407.21248/) (2024)
- [LUMIA: Linear probing for Unimodal and MultiModal Membership Inference Attacks leveraging internal LLM states](https://mlsanity.com/paper/2411.19876/) (2024)
- [Unveiling the Unseen: Exploring Whitebox Membership Inference through the Lens of Explainability](https://mlsanity.com/paper/2407.01306/) (2024)
- [The Heap: A Contamination-Free Multilingual Code Dataset for Evaluating Large Language Models](https://mlsanity.com/paper/2501.09653/) (2025)

## Abstract (arXiv)

As Large Language Models (LLMs) for code increasingly utilize massive, often non-permissively licensed datasets, evaluating data contamination through Membership Inference Attacks (MIAs) has become critical. We propose SERSEM (Selective Entropy-Weighted Scoring for Membership Inference), a novel white-box attack framework that suppresses uninformative syntactical boilerplate to amplify specific memorization signals. SERSEM utilizes a dual-signal methodology: first, a continuous character-level weight mask is derived through static Abstract Syntax Tree (AST) analysis, spellchecking-based multilingual logic detection, and offline linting. Second, these heuristic weights are used to pool internal transformer activations and calibrate token-level Z-scores from the output logits. Evaluated on a 25,000-sample balanced dataset, SERSEM achieves a global AUC-ROC of 0.7913 on the StarCoder2-3B model and 0.7867 on the StarCoder2-7B model, consistently outperforming the implemented probability-based baselines Loss, Min-K% Prob, and PAC. Our findings demonstrate that focusing on human-centric coding anomalies provides a significantly more robust indicator of verbatim memorization than sequence-level probability averages.