# Which Alert Removals are Beneficial?

arXiv: https://arxiv.org/abs/2603.21322 · HTML: https://mlsanity.com/paper/2603.21322/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 77 references that are in the map.
- First posted on arXiv 22 Mar 2026.

## Builds on (most cited first, 16 of 16)

- [A Closer Look at Memorization in Deep Networks](https://mlsanity.com/paper/1706.05394/) (2017)
- [Data Programming: Creating Large Training Sets, Quickly](https://mlsanity.com/paper/1605.07723/) (2016)
- [Applied Causal Inference Powered by ML and AI](https://mlsanity.com/paper/2403.02467/) (2024)
- [A Selective Review of Negative Control Methods in Epidemiology](https://mlsanity.com/paper/2009.05641/) (2020)
- [A Fine-grained Data Set and Analysis of Tangling in Bug Fixing Commits](https://mlsanity.com/paper/2011.06244/) (2020)
- [Why We Refactor? Confessions of GitHub Contributors](https://mlsanity.com/paper/1607.02459/) (2016)
- [Does Code Quality Affect Pull Request Acceptance? An empirical study](https://mlsanity.com/paper/1908.09321/) (2019)
- [Challenges in Statistics: A Dozen Challenges in Causality and Causal Inference](https://mlsanity.com/paper/2508.17099/) (2025)
- [Are SonarQube Rules Inducing Bugs?](https://mlsanity.com/paper/1907.00376/) (2019)
- [Follow Your Nose -- Which Code Smells are Worth Chasing?](https://mlsanity.com/paper/2103.01861/) (2021)
- [ComSum: Commit Messages Summarization and Meaning Preservation](https://mlsanity.com/paper/2108.10763/) (2021)
- [A Longitudinal Study of Static Analysis Warning Evolution and the Effects of PMD on Software Quality in Apache Open Source Projects](https://mlsanity.com/paper/1912.02179/) (2019)
- [End to End Software Engineering Research](https://mlsanity.com/paper/2112.11858/) (2021)
- [Automated Refactoring of Nested-IF Formulae in Spreadsheets](https://mlsanity.com/paper/1712.09797/) (2017)
- [An Experimental Evaluation of a De-biasing Intervention for Professional Software Developers](https://mlsanity.com/paper/1804.03919/) (2018)
- [Behind the Intent of Extract Method Refactoring: A Systematic Literature Review](https://mlsanity.com/paper/2312.12600/) (2023)

## Abstract (arXiv)

Context: Static analysis captures software engineering knowledge and alerts on possibly problematic patterns. Previous work showed that they indeed have predictive power for various problems. However, the impact of removing the alerts is unclear. Aim: We would like to evaluate the impact of alert removals on code complexity and the tendency to bugs. Method: We evaluate the impact of removing alerts using three complementary methods. 1. We conducted a randomized controlled trial and built a dataset of 521 manual alert-removing interventions 2. We profiled intervention-like events using labeling functions. We applied these labeling functions to code commits, found intervention-like natural events, and used them to analyze the impact on the tendency to bugs. 3. We built a dataset of 8,245 alert removals, more than 15 times larger than our dataset of manual interventions. We applied supervised learning to the alert removals, aiming to predict their impact on the tendency to bugs. Results: We identified complexity-reducing interventions that reduce the probability of future bugs. Such interventions are relevant to 33% of Python files and might reduce the tendency to bugs by 5.5 percentage points. Conclusions: We presented methods to evaluate the impact of interventions. The methods can identify a large number of natural interventions that are highly needed in causality research in many domains.