mlsanityData to 7 Oct 2026

Is ChatGPT better than Human Annotators? Potential and Limitations of ChatGPT in Explaining Implicit Hate Speech

arXiv 2302.07736 ↗First posted 11 Feb 2023cs.CL cs.HCcited by 165

Not on the map. Why?

Abstract (arXiv)

Recent studies have alarmed that many online hate speeches are implicit. With its subtle nature, the explainability of the detection of such hateful speech has been a challenging problem. In this work, we examine whether ChatGPT can be used for providing natural language explanations (NLEs) for implicit hateful speech detection. We design our prompt to elicit concise ChatGPT-generated NLEs and conduct user studies to evaluate their qualities by comparison with human-written NLEs. We discuss the potential and limitations of ChatGPT in the context of implicit hateful speech research.

Key facts. Cited by 165 papers in the map (data through 7 Oct 2026). Builds on 13 of its 21 references that are in the map. First posted on arXiv 11 Feb 2023.

Builds on

13 of 21 references.

References · 13

  1. 2023How Close is ChatGPT to Human Experts? Comparison Corpus, Evaluation, and Detectioncited by 472
  2. 2018e-SNLI: Natural Language Inference with Natural Language Explanationscited by 468
  3. 2019Social Bias Frames: Reasoning about Social and Power Implications of Languagecited by 377
  4. 2021Latent Hatred: A Benchmark for Understanding Implicit Hate Speechcited by 216
  5. 2018Hate Speech Dataset from a White Supremacy Forumcited by 196
  6. 2023An Analysis of the Automatic Bug Fixing Performance of ChatGPTcited by 108
  7. 2022ChatGPT Makes Medicine Easy to Swallow: An Exploratory Case Study on Simplified Radiology Reportscited by 100
  8. 2019Evaluating the State-of-the-Art of End-to-End Natural Language Generation: The E2E NLG Challengecited by 86
  9. 2022How would Stance Detection Techniques Evolve after the Launch of ChatGPT?cited by 76
  10. 2022Chain of Explanation: New Prompting Method to Generate Higher Quality Natural Language Explanation for Implicit Hate Speechcited by 25
  11. 2016Kek, Cucks, and God Emperor Trump: A Measurement Study of 4chan's Politically Incorrect Forum and Its Effects on the Webcited by 15
  12. 2021Predicting Anti-Asian Hateful Users on Twitter during COVID-19cited by 11
1 more
  1. 2015Exploring Cyberbullying and Other Toxic Behavior in Team Competition Online Gamescited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 53; 2024: 67; 2025: 28; 2026: 17.

165 papers.

Sort:

By area:

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  3. 2023ChatGPT Outperforms Crowd-Workers for Text-Annotation Taskscited by 849
  4. 2023Can Large Language Models Be an Alternative to Human Evaluations?cited by 729
  5. 2023Is ChatGPT a Good NLG Evaluator? A Preliminary Studycited by 420
  6. 2023Learning to Compress Prompts with Gist Tokenscited by 379
  7. 2024Leak, Cheat, Repeat: Data Contamination and Evaluation Malpractices in Closed-Source LLMscited by 224
  8. 2023Summary of ChatGPT-Related Research and Perspective Towards the Future of Large Language Modelscited by 152
  9. 2023Evaluating ChatGPT's Information Extraction Capabilities: An Assessment of Performance, Explainability, Calibration, and Faithfulnesscited by 130
  10. 2023A Systematic Study and Comprehensive Evaluation of ChatGPT on Benchmark Datasetscited by 123
  11. 2023FollowBench: A Multi-level Fine-grained Constraints Following Benchmark for Large Language Modelscited by 112
  12. 2023LawBench: Benchmarking Legal Knowledge of Large Language Modelscited by 111
38 more
  1. 2024Critique-out-Loud Reward Modelscited by 105
  2. 2024Best Practices and Lessons Learned on Synthetic Datacited by 104
  3. 2023CoAnnotating: Uncertainty-Guided Work Allocation between Human and Large Language Models for Data Annotationcited by 93
  4. 2023Can ChatGPT Reproduce Human-Generated Labels? A Study of Social Computing Taskscited by 88
  5. 2023Training Socially Aligned Language Models on Simulated Social Interactionscited by 77
  6. 2023Can ChatGPT Replace Traditional KBQA Models? An In-depth Analysis of the Question Answering Performance of the GPT LLM Familycited by 71
  7. 2023Let the LLMs Talk: Simulating Human-to-Human Conversational QA via Zero-Shot LLM-to-LLM Interactionscited by 70
  8. 2024What's in the Image? A Deep-Dive into the Vision of Vision Language Modelscited by 66
  9. 2024Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systemscited by 63
  10. 2023Testing the Reliability of ChatGPT for Text Annotation and Classification: A Cautionary Remarkcited by 54
  11. 2023Probing LLMs for hate speech detection: strengths and vulnerabilitiescited by 52
  12. 2024GOAT-Bench: Safety Insights to Large Multimodal Models through Meme-Based Social Abusecited by 50
  13. 2024Explainable Fake News Detection With Large Language Model via Defense Among Competing Wisdomcited by 49
  14. 2023ChatGPT: Beginning of an End of Manual Linguistic Data Annotation? Use Case of Automatic Genre Identificationcited by 45
  15. 2023"HOT" ChatGPT: The promise of ChatGPT in detecting and discriminating hateful, offensive, and toxic comments on social mediacited by 45
  16. 2024Usable XAI: 10 Strategies Towards Exploiting Explainability in the LLM Eracited by 45
  17. 2023Discrete Prompt Compression with Reinforcement Learningcited by 42
  18. 2023CHEAT: A Large-scale Dataset for Detecting ChatGPT-writtEn AbsTractscited by 39
  19. 2024If in a Crowdsourced Data Annotation Pipeline, a GPT-4cited by 38
  20. 2024Leveraging Large Language Models for NLG Evaluation: Advances and Challengescited by 37
  21. 2024Graph-enhanced Large Language Models in Asynchronous Plan Reasoningcited by 37
  22. 2025Neither Valid nor Reliable? Investigating the Use of LLMs as Judgescited by 37
  23. 2024Decoding Biases: Automated Methods and LLM Judges for Gender Bias Detection in Language Modelscited by 33
  24. 2023Can GPT-4 Support Analysis of Textual Data in Tasks Requiring Highly Specialized Domain Expertise?cited by 30
  25. 2024Content Moderation by LLM: From Accuracy to Legitimacycited by 30
  26. 2024AI for social science and social science of AI: A Surveycited by 29
  27. 2024The Effectiveness of LLMs as Annotators: A Comparative Overview and Empirical Analysis of Direct Representationcited by 29
  28. 2023Assessing Hidden Risks of LLMs: An Empirical Study on Robustness, Consistency, and Credibilitycited by 28
  29. 2023Embedding Democratic Values into Social Media AIs via Societal Objective Functionscited by 25
  30. 2024Don't Go To Extremes: Revealing the Excessive Sensitivity and Calibration Limitations of LLMs in Implicit Hate Speech Detectioncited by 25
  31. 2024What Does the Bot Say? Opportunities and Risks of Large Language Models in Social Media Bot Detectioncited by 24
  32. 2024CLAVE: An Adaptive Framework for Evaluating Values of LLM Generated Responsescited by 23
  33. 2024Hate Personified: Investigating the role of LLMs in content moderationcited by 23
  34. 2024Enhancing Text Classification through LLM-Driven Active Learning and Human Annotationcited by 20
  35. 2023Large Language Models for Propaganda Span Annotationcited by 19
  36. 2024Open Source Language Models Can Provide Feedback: Evaluating LLMs' Ability to Help Students Using GPT-4-As-A-Judgecited by 19
  37. 2023Decoding ChatGPT: A Taxonomy of Existing Research, Current Challenges, and Possible Future Directionscited by 18
  38. 2023Automated Claim Matching with Large Language Models: Empowering Fact-Checkers in the Fight Against Misinformationcited by 18

50 most cited of 165 All 165 (JSON)