# BiasRV: Uncovering Biased Sentiment Predictions at Runtime

arXiv: https://arxiv.org/abs/2105.14874 · HTML: https://mlsanity.com/paper/2105.14874/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 May 2021.

## Built on by (most cited first, 12 of 12)

- [Stealthy Backdoor Attack for Code Models](https://mlsanity.com/paper/2301.02496/) (2023)
- [Towards Fair Machine Learning Software: Understanding and Addressing Model Bias Through Counterfactual Thinking](https://mlsanity.com/paper/2302.08018/) (2023)
- [Towards an Enhanced Understanding of Bias in Pre-trained Neural Language Models: A Survey with Special Emphasis on Affective Bias](https://mlsanity.com/paper/2204.10365/) (2022)
- [Do Existing Testing Tools Really Uncover Gender Bias in Text-to-Image Models?](https://mlsanity.com/paper/2501.15775/) (2025)
- [RAI4IoE: Responsible AI for Enabling the Internet of Energy](https://mlsanity.com/paper/2309.11691/) (2023)
- [Unbiasing on the Fly: Explanation-Guided Human Oversight of Machine Learning System Decisions](https://mlsanity.com/paper/2406.17906/) (2024)
- [Blacks is to Anger as Whites is to Joy? Understanding Latent Affective Bias in Large Pre-trained Neural Language Models](https://mlsanity.com/paper/2301.09003/) (2023)
- [Distribution-aware Fairness Test Generation](https://mlsanity.com/paper/2305.13935/) (2023)
- [Efficient DNN-Powered Software with Fair Sparse Models](https://mlsanity.com/paper/2407.02805/) (2024)
- [Automated Harmfulness Testing for Code Large Language Models](https://mlsanity.com/paper/2503.16740/) (2025)
- [Applications and Challenges of Fairness APIs in Machine Learning Software](https://mlsanity.com/paper/2508.16377/) (2025)
- [Ethics Testing: Proactive Identification of Generative AI System Harms](https://mlsanity.com/paper/2604.22089/) (2026)

## Abstract (arXiv)

Sentiment analysis (SA) systems, though widely applied in many domains, have been demonstrated to produce biased results. Some research works have been done in automatically generating test cases to reveal unfairness in SA systems, but the community still lacks tools that can monitor and uncover biased predictions at runtime. This paper fills this gap by proposing BiasRV, the first tool to raise an alarm when a deployed SA system makes a biased prediction on a given input text. To implement this feature, BiasRV dynamically extracts a template from an input text and from the template generates gender-discriminatory mutants (semantically-equivalent texts that only differ in gender information). Based on popular metrics used to evaluate the overall fairness of an SA system, we define distributional fairness property for an individual prediction of an SA system. This property specifies a requirement that for one piece of text, mutants from different gender classes should be treated similarly as a whole. Verifying the distributional fairness property causes much overhead to the running system. To run more efficiently, BiasRV adopts a two-step heuristic: (1) sampling several mutants from each gender and checking if the system predicts them as of the same sentiment, (2) checking distributional fairness only when sampled mutants have conflicting results. Experiments show that compared to directly checking the distributional fairness property for each input text, our two-step heuristic can decrease overhead used for analyzing mutants by 73.81% while only resulting in 6.7% of biased predictions being missed. Besides, BiasRV can be used conveniently without knowing the implementation of SA systems. Future researchers can easily extend BiasRV to detect more types of bias, e.g. race and occupation.