# Less, but Stronger: On the Value of Strong Heuristics in Semi-supervised Learning for Software Analytics

arXiv: https://arxiv.org/abs/2302.01997 · HTML: https://mlsanity.com/paper/2302.01997/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 23 of its 116 references that are in the map.
- First posted on arXiv 3 Feb 2023.

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Semantics derived automatically from language corpora contain human-like biases](https://mlsanity.com/paper/1608.07187/) (2016)
- [Certifying and removing disparate impact](https://mlsanity.com/paper/1412.3756/) (2014)
- [AI Fairness 360: An Extensible Toolkit for Detecting, Understanding, and Mitigating Unwanted Algorithmic Bias](https://mlsanity.com/paper/1810.01943/) (2018)
- [Deep Co-Training for Semi-Supervised Image Recognition](https://mlsanity.com/paper/1803.05984/) (2018)
- [Fairness Testing: Testing Software for Discrimination](https://mlsanity.com/paper/1709.03221/) (2017)
- [Discrimination in Online Ad Delivery](https://mlsanity.com/paper/1301.6822/) (2013)
- [Bias in Machine Learning Software: Why? How? What to do?](https://mlsanity.com/paper/2105.12195/) (2021)
- [Racial Disparity in Natural Language Processing: A Case Study of Social Media African-American English](https://mlsanity.com/paper/1707.00061/) (2017)
- [Tuning for Software Analytics: is it Really Necessary?](https://mlsanity.com/paper/1609.01759/) (2016)
- [Is "Better Data" Better than "Better Data Miners"? (On the Benefits of Tuning SMOTE for Defect Prediction)](https://mlsanity.com/paper/1705.03697/) (2017)
- [Detecting False Alarms from Automatic Static Analysis Tools: How Far are We?](https://mlsanity.com/paper/2202.05982/) (2022)
- [A Benchmark Study on Sentiment Analysis for Software Engineering Research](https://mlsanity.com/paper/1803.06525/) (2018)
- [Better Data Labelling with EMBLEM (and how that Impacts Defect Prediction)](https://mlsanity.com/paper/1905.01719/) (2019)
- [How to "DODGE" Complex Software Analytics?](https://mlsanity.com/paper/1902.01838/) (2019)
- [Improving Vulnerability Inspection Efficiency Using Active Learning](https://mlsanity.com/paper/1803.06545/) (2018)
- [Identifying Self-Admitted Technical Debts with Jitterbug: A Two-step Approach](https://mlsanity.com/paper/2002.11049/) (2020)
- [FRUGAL: Unlocking SSL for Software Analytics](https://mlsanity.com/paper/2108.09847/) (2021)
- [Replication Can Improve Prior Results: A GitHub Study of Pull Request Acceptance](https://mlsanity.com/paper/1902.04060/) (2019)
- [Learning to Recognize Actionable Static Code Warnings (is Intrinsically Easy)](https://mlsanity.com/paper/2006.00444/) (2020)
- [Defect Reduction Planning (using TimeLIME)](https://mlsanity.com/paper/2006.07416/) (2020)
- [FLASH: A Faster Optimizer for SBSE Tasks](https://mlsanity.com/paper/1705.05018/) (2017)
- [TERMINATOR: Better Automated UI Test Case Prioritization](https://mlsanity.com/paper/1905.07019/) (2019)

## Abstract (arXiv)

In many domains, there are many examples and far fewer labels for those examples; e.g. we may have access to millions of lines of source code, but access to only a handful of warnings about that code. In those domains, semi-supervised learners (SSL) can extrapolate labels from a small number of examples to the rest of the data. Standard SSL algorithms use ``weak'' knowledge (i.e. those not based on specific SE knowledge) such as (e.g.) co-train two learners and use good labels from one to train the other. Another approach of SSL in software analytics is potentially use ``strong'' knowledge that use SE knowledge. For example, an often-used heuristic in SE is that unusually large artifacts contain undesired properties (e.g. more bugs). This paper argues that such ``strong'' algorithms perform better than those standard, weaker, SSL algorithms. We show this by learning models from labels generated using weak SSL or our ``stronger'' FRUGAL algorithm. In four domains (distinguishing security-related bug reports; mitigating bias in decision-making; predicting issue close time; and (reducing false alarms in static code warnings), FRUGAL required only 2.5% of the data to be labeled yet out-performed standard semi-supervised learners that relied on (e.g.) some domain-independent graph theory concepts. Hence, for future work, we strongly recommend the use of strong heuristics for semi-supervised learning for SE applications. To better support other researchers, our scripts and data are on-line at https://github.com/HuyTu7/FRUGAL.