# Probing Model Signal-Awareness via Prediction-Preserving Input Minimization

arXiv: https://arxiv.org/abs/2011.14934 · HTML: https://mlsanity.com/paper/2011.14934/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Nov 2020.

## Built on by (most cited first, 14 of 14)

- [Reducing Cost of LLM Agents with Trajectory Reduction](https://mlsanity.com/paper/2509.23586/) (2025)
- [Counterfactual Explanations for Models of Code](https://mlsanity.com/paper/2111.05711/) (2021)
- [Understanding Neural Code Intelligence Through Program Simplification](https://mlsanity.com/paper/2106.03353/) (2021)
- [Memorization and Generalization in Neural Code Intelligence Models](https://mlsanity.com/paper/2106.08704/) (2021)
- [Graph Neural Networks for Vulnerability Detection: A Counterfactual Explanation](https://mlsanity.com/paper/2404.15687/) (2024)
- [The Hidden Cost of Readability: How Code Formatting Silently Consumes Your LLM Budget](https://mlsanity.com/paper/2508.13666/) (2025)
- [Syntax-Guided Program Reduction for Understanding Neural Code Intelligence Models](https://mlsanity.com/paper/2205.14374/) (2022)
- [Code2Snapshot: Using Code Snapshots for Learning Representations of Source Code](https://mlsanity.com/paper/2111.01097/) (2021)
- [Study of Distractors in Neural Models of Code](https://mlsanity.com/paper/2303.01739/) (2023)
- [Data-Driven AI Model Signal-Awareness Enhancement and Introspection](https://mlsanity.com/paper/2111.05827/) (2021)
- [Redundancy and Concept Analysis for Code-trained Language Models](https://mlsanity.com/paper/2305.00875/) (2023)
- [Extracting Label-specific Key Input Features for Neural Code Intelligence Models](https://mlsanity.com/paper/2202.06474/) (2022)
- [LEANCODE: Understanding Models Better for Code Simplification of Pre-trained Large Language Models](https://mlsanity.com/paper/2505.14759/) (2025)
- [From User Interface to Agent Interface: Efficiency Optimization of UI Representations for LLM Agents](https://mlsanity.com/paper/2512.13438/) (2025)

## Abstract (arXiv)

This work explores the signal awareness of AI models for source code understanding. Using a software vulnerability detection use case, we evaluate the models' ability to capture the correct vulnerability signals to produce their predictions. Our prediction-preserving input minimization (P2IM) approach systematically reduces the original source code to a minimal snippet which a model needs to maintain its prediction. The model's reliance on incorrect signals is then uncovered when the vulnerability in the original code is missing in the minimal snippet, both of which the model however predicts as being vulnerable. We measure the signal awareness of models using a new metric we propose- Signal-aware Recall (SAR). We apply P2IM on three different neural network architectures across multiple datasets. The results show a sharp drop in the model's Recall from the high 90s to sub-60s with the new metric, highlighting that the models are presumably picking up a lot of noise or dataset nuances while learning their vulnerability detection logic. Although the drop in model performance may be perceived as an adversarial attack, but this isn't P2IM's objective. The idea is rather to uncover the signal-awareness of a black-box model in a data-driven manner via controlled queries. SAR's purpose is to measure the impact of task-agnostic model training, and not to suggest a shortcoming in the Recall metric. The expectation, in fact, is for SAR to match Recall in the ideal scenario where the model truly captures task-specific signals.