# Modeling Users and Online Communities for Abuse Detection: A Position on Ethics and Explainability

arXiv: https://arxiv.org/abs/2103.17191 · HTML: https://mlsanity.com/paper/2103.17191/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 72 references that are in the map.
- First posted on arXiv 31 Mar 2021.

## Builds on (most cited first, 28 of 28)

- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [node2vec: Scalable Feature Learning for Networks](https://mlsanity.com/paper/1607.00653/) (2016)
- [Explanation in Artificial Intelligence: Insights from the Social Sciences](https://mlsanity.com/paper/1706.07269/) (2017)
- [GNNExplainer: Generating Explanations for Graph Neural Networks](https://mlsanity.com/paper/1903.03894/) (2019)
- [Explaining Explanations: An Overview of Interpretability of Machine Learning](https://mlsanity.com/paper/1806.00069/) (2018)
- [Social Bias Frames: Reasoning about Social and Power Implications of Language](https://mlsanity.com/paper/1911.03891/) (2019)
- [Large Scale Crowdsourcing and Characterization of Twitter Abusive Behavior](https://mlsanity.com/paper/1802.00393/) (2018)
- [Ex Machina: Personal Attacks Seen at Scale](https://mlsanity.com/paper/1610.08914/) (2016)
- [Understanding Abuse: A Typology of Abusive Language Detection Subtasks](https://mlsanity.com/paper/1705.09899/) (2017)
- [The Hidden Assumptions Behind Counterfactual Explanations and Principal Reasons](https://mlsanity.com/paper/1912.04930/) (2019)
- [The Dangers of Post-hoc Interpretability: Unjustified Counterfactual Explanations](https://mlsanity.com/paper/1907.09294/) (2019)
- [Measuring the Reliability of Hate Speech Annotations: The Case of the European Refugee Crisis](https://mlsanity.com/paper/1701.08118/) (2017)
- [Detecting Online Hate Speech Using Context Aware Models](https://mlsanity.com/paper/1710.07395/) (2017)
- [Characterizing and Detecting Hateful Users on Twitter](https://mlsanity.com/paper/1803.08977/) (2018)
- [A Just and Comprehensive Strategy for Using NLP to Address Online Abuse](https://mlsanity.com/paper/1906.01738/) (2019)
- [One-step and Two-step Classification for Abusive Language Detection on Twitter](https://mlsanity.com/paper/1706.01206/) (2017)
- [Leveraging Intra-User and Inter-User Representation Learning for Automated Hate Speech Detection](https://mlsanity.com/paper/1804.03124/) (2018)
- [Abusive Language Detection with Graph Convolutional Networks](https://mlsanity.com/paper/1904.04073/) (2019)
- [You Shall Know a User by the Company It Keeps: Dynamic Representations for Social Media Users in NLP](https://mlsanity.com/paper/1909.00412/) (2019)
- [Joint Modelling of Emotion and Abusive Language Detection](https://mlsanity.com/paper/2005.14028/) (2020)
- [Improved Abusive Comment Moderation with User Embeddings](https://mlsanity.com/paper/1708.03699/) (2017)
- [The Road to Success: Assessing the Fate of Linguistic Innovations in Online Communities](https://mlsanity.com/paper/1806.05838/) (2018)
- [Attending the Emotions to Detect Online Abusive Language](https://mlsanity.com/paper/1909.03100/) (2019)
- [Graph embeddings for Abusive Language Detection](https://mlsanity.com/paper/2101.02988/) (2021)

## Built on by (most cited first, 4 of 4)

- [TwiBot-22: Towards Graph-Based Twitter Bot Detection](https://mlsanity.com/paper/2206.04564/) (2022)
- [The State of Profanity Obfuscation in Natural Language Processing](https://mlsanity.com/paper/2210.07595/) (2022)
- [Yesterday's News: Benchmarking Multi-Dimensional Out-of-Distribution Generalization of Misinformation Detection Models](https://mlsanity.com/paper/2410.18122/) (2024)
- [A (More) Realistic Evaluation Setup for Generalisation of Community Models on Malicious Content Detection](https://mlsanity.com/paper/2404.01822/) (2024)

## Abstract (arXiv)

Abuse on the Internet is an important societal problem of our time. Millions of Internet users face harassment, racism, personal attacks, and other types of abuse across various platforms. The psychological effects of abuse on individuals can be profound and lasting. Consequently, over the past few years, there has been a substantial research effort towards automated abusive language detection in the field of NLP. In this position paper, we discuss the role that modeling of users and online communities plays in abuse detection. Specifically, we review and analyze the state of the art methods that leverage user or community information to enhance the understanding and detection of abusive language. We then explore the ethical challenges of incorporating user and community information, laying out considerations to guide future research. Finally, we address the topic of explainability in abusive language detection, proposing properties that an explainable method should aim to exhibit. We describe how user and community information can facilitate the realization of these properties and discuss the effective operationalization of explainability in view of the properties.