# Justice in Misinformation Detection Systems: An Analysis of Algorithms, Stakeholders, and Potential Harms

arXiv: https://arxiv.org/abs/2204.13568 · HTML: https://mlsanity.com/paper/2204.13568/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 81 references that are in the map.
- First posted on arXiv 28 Apr 2022.

## Builds on (most cited first, 20 of 20)

- [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016)
- [FEVER: a large-scale dataset for Fact Extraction and VERification](https://mlsanity.com/paper/1803.05355/) (2018)
- [Language (Technology) is Power: A Critical Survey of "Bias" in NLP](https://mlsanity.com/paper/2005.14050/) (2020)
- [A Survey on Automated Fact-Checking](https://mlsanity.com/paper/2108.11896/) (2021)
- [Bias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Setting](https://mlsanity.com/paper/1901.09451/) (2019)
- [Dealing with Disagreements: Looking Beyond the Majority Vote in Subjective Annotations](https://mlsanity.com/paper/2110.05719/) (2021)
- [Measurement and Fairness](https://mlsanity.com/paper/1912.05511/) (2019)
- [Directions in Abusive Language Training Data: Garbage In, Garbage Out](https://mlsanity.com/paper/2004.01670/) (2020)
- [That is a Known Lie: Detecting Previously Fact-Checked Claims](https://mlsanity.com/paper/2005.06058/) (2020)
- [Fighting the COVID-19 Infodemic: Modeling the Perspective of Journalists, Fact-Checkers, Social Media Platforms, Policy Makers, and the Society](https://mlsanity.com/paper/2005.00033/) (2020)
- [Towards Automated Factchecking: Developing an Annotation Schema and Benchmark for Consistent Automated Claim Detection](https://mlsanity.com/paper/1809.08193/) (2018)
- [Overview of CheckThat! 2020: Automatic Identification and Verification of Claims in Social Media](https://mlsanity.com/paper/2007.07997/) (2020)
- [Fairness Under Composition](https://mlsanity.com/paper/1806.06122/) (2018)
- [Diversity and Inclusion Metrics in Subset Selection](https://mlsanity.com/paper/2002.03256/) (2020)
- [When Fair Ranking Meets Uncertain Inference](https://mlsanity.com/paper/2105.02091/) (2021)
- [Hierarchical Multi-head Attentive Network for Evidence-aware Fake News Detection](https://mlsanity.com/paper/2102.02680/) (2021)
- [FAKTA: An Automatic End-to-End Fact Checking System](https://mlsanity.com/paper/1906.04164/) (2019)
- [Diversity in Sociotechnical Machine Learning Systems](https://mlsanity.com/paper/2107.09163/) (2021)
- [Hate Speech Classifiers Learn Human-Like Social Stereotypes](https://mlsanity.com/paper/2110.14839/) (2021)
- [RP-DNN: A Tweet level propagation context based deep neural networks for early rumor detection in Social Media](https://mlsanity.com/paper/2002.12683/) (2020)

## Built on by (most cited first, 10 of 10)

- [Underspecification in Scene Description-to-Depiction Tasks](https://mlsanity.com/paper/2210.05815/) (2022)
- [Beyond Behaviorist Representational Harms: A Plan for Measurement and Mitigation](https://mlsanity.com/paper/2402.01705/) (2024)
- [Epistemic Injustice in Generative AI](https://mlsanity.com/paper/2408.11441/) (2024)
- [Community Moderation and the New Epistemology of Fact Checking on Social Media](https://mlsanity.com/paper/2505.20067/) (2025)
- [MiDe22: An Annotated Multi-Event Tweet Dataset for Misinformation Detection](https://mlsanity.com/paper/2210.05401/) (2022)
- [More Data Can Lead Us Astray: Active Data Acquisition in the Presence of Label Bias](https://mlsanity.com/paper/2207.07723/) (2022)
- [Secondary Stakeholders in AI: Fighting for, Brokering, and Navigating Agency](https://mlsanity.com/paper/2506.07281/) (2025)
- [Fairness-Aware Multi-Group Target Detection in Online Discussion](https://mlsanity.com/paper/2407.11933/) (2024)
- [Graphical Models of False Information and Fact Checking Ecosystems](https://mlsanity.com/paper/2208.11582/) (2022)
- [Weak Supervision for Real World Graphs](https://mlsanity.com/paper/2506.02451/) (2025)

## Abstract (arXiv)

Faced with the scale and surge of misinformation on social media, many platforms and fact-checking organizations have turned to algorithms for automating key parts of misinformation detection pipelines. While offering a promising solution to the challenge of scale, the ethical and societal risks associated with algorithmic misinformation detection are not well-understood. In this paper, we employ and extend upon the notion of informational justice to develop a framework for explicating issues of justice relating to representation, participation, distribution of benefits and burdens, and credibility in the misinformation detection pipeline. Drawing on the framework: (1) we show how injustices materialize for stakeholders across three algorithmic stages in the pipeline; (2) we suggest empirical measures for assessing these injustices; and (3) we identify potential sources of these harms. This framework should help researchers, policymakers, and practitioners reason about potential harms or risks associated with these algorithms and provide conceptual guidance for the design of algorithmic fairness audits in this domain.