# HarmPot: An Annotation Framework for Evaluating Offline Harm Potential of Social Media Text

arXiv: https://arxiv.org/abs/2403.11108 · HTML: https://mlsanity.com/paper/2403.11108/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 132 references that are in the map.
- First posted on arXiv 17 Mar 2024.

## Builds on (most cited first, 30 of 30)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [The State and Fate of Linguistic Diversity and Inclusion in the NLP World](https://mlsanity.com/paper/2004.09095/) (2020)
- [Automated Hate Speech Detection and the Problem of Offensive Language](https://mlsanity.com/paper/1703.04009/) (2017)
- [Predicting the Type and Target of Offensive Posts in Social Media](https://mlsanity.com/paper/1902.09666/) (2019)
- [SemEval-2019 Task 6: Identifying and Categorizing Offensive Language in Social Media (OffensEval)](https://mlsanity.com/paper/1903.08983/) (2019)
- [Deep Learning for Hate Speech Detection in Tweets](https://mlsanity.com/paper/1706.00188/) (2017)
- [Directions in Abusive Language Training Data: Garbage In, Garbage Out](https://mlsanity.com/paper/2004.01670/) (2020)
- [Multilingual and Multi-Aspect Hate Speech Analysis](https://mlsanity.com/paper/1908.11049/) (2019)
- [Understanding Abuse: A Typology of Abusive Language Detection Subtasks](https://mlsanity.com/paper/1705.09899/) (2017)
- [CONAN -- COunter NArratives through Nichesourcing: a Multilingual Dataset of Responses to Fight Online Hate Speech](https://mlsanity.com/paper/1910.03270/) (2019)
- [Measuring the Reliability of Hate Speech Annotations: The Case of the European Refugee Crisis](https://mlsanity.com/paper/1701.08118/) (2017)
- [SOLID: A Large-Scale Semi-Supervised Dataset for Offensive Language Identification](https://mlsanity.com/paper/2004.14454/) (2020)
- [Beyond Fair Pay: Ethical Implications of NLP Crowdsourcing](https://mlsanity.com/paper/2104.10097/) (2021)
- [A Just and Comprehensive Strategy for Using NLP to Address Online Abuse](https://mlsanity.com/paper/1906.01738/) (2019)
- [Detecting Hate Speech in Social Media](https://mlsanity.com/paper/1712.06427/) (2017)
- [Aggression-annotated Corpus of Hindi-English Code-mixed Data](https://mlsanity.com/paper/1803.09402/) (2018)
- [Overview of the HASOC track at FIRE 2020: Hate Speech and Offensive Content Identification in Indo-European Languages](https://mlsanity.com/paper/2108.05927/) (2021)
- [Hate Speech detection in the Bengali language: A dataset and its baseline evaluation](https://mlsanity.com/paper/2012.09686/) (2020)
- [Developing a Multilingual Annotated Corpus of Misogyny and Aggression](https://mlsanity.com/paper/2003.07428/) (2020)
- [Classification Benchmarks for Under-resourced Bengali Language based on Multichannel Convolutional-LSTM Network](https://mlsanity.com/paper/2004.07807/) (2020)
- [DeepHateExplainer: Explainable Hate Speech Detection in Under-resourced Bengali Language](https://mlsanity.com/paper/2012.14353/) (2020)
- [Overview of the HASOC Subtrack at FIRE 2021: Hate Speech and Offensive Content Identification in English and Indo-Aryan Languages](https://mlsanity.com/paper/2112.09301/) (2021)
- [Detecting weak and strong Islamophobic hate speech on social media](https://mlsanity.com/paper/1812.10400/) (2018)
- [BD-SHS: A Benchmark Dataset for Learning to Detect Online Bangla Hate Speech in Different Social Contexts](https://mlsanity.com/paper/2206.00372/) (2022)
- [Galileo at SemEval-2020 Task 12: Multi-lingual Learning for Offensive Language Identification using Pre-trained Language Models](https://mlsanity.com/paper/2010.03542/) (2020)
- [Bangla hate speech detection on social media using attention-based recurrent neural network](https://mlsanity.com/paper/2203.16775/) (2022)
- [Overview of Abusive and Threatening Language Detection in Urdu at FIRE 2021](https://mlsanity.com/paper/2207.06710/) (2022)
- [HS-BAN: A Benchmark Dataset of Social Media Comments for Hate Speech Detection in Bangla](https://mlsanity.com/paper/2112.01902/) (2021)

## Built on by (most cited first, 1 of 1)

- [NJUST-KMG at TRAC-2024 Tasks 1 and 2: Offline Harm Potential Identification](https://mlsanity.com/paper/2403.19713/) (2024)

## Abstract (arXiv)

In this paper, we discuss the development of an annotation schema to build datasets for evaluating the offline harm potential of social media texts. We define "harm potential" as the potential for an online public post to cause real-world physical harm (i.e., violence). Understanding that real-world violence is often spurred by a web of triggers, often combining several online tactics and pre-existing intersectional fissures in the social milieu, to result in targeted physical violence, we do not focus on any single divisive aspect (i.e., caste, gender, religion, or other identities of the victim and perpetrators) nor do we focus on just hate speech or mis/dis-information. Rather, our understanding of the intersectional causes of such triggers focuses our attempt at measuring the harm potential of online content, irrespective of whether it is hateful or not. In this paper, we discuss the development of a framework/annotation schema that allows annotating the data with different aspects of the text including its socio-political grounding and intent of the speaker (as expressed through mood and modality) that together contribute to it being a trigger for offline harm. We also give a comparative analysis and mapping of our framework with some of the existing frameworks.