You Frame It: How Conceptual Representations Shape LLM Detection and Reasoning about Antisemitism
On the map: LLM reasoning → Hallucination detection and mitigation
Abstract (arXiv)
LLMs enable the integration of external conceptual resources at inference time, creating new opportunities for detecting ideologically and historically complex phenomena such as antisemitism. We investigate how different forms of conceptual grounding affect antisemitism detection and explanation behavior across four state-of-the-art LLMs. Using two expert-annotated datasets, we compare definitional, fine-grained taxonomic, example-augmented, and large-context representations of antisemitism. We find that fine-grained taxonomic representations substantially improve recall, while simultaneously reducing precision. Surprisingly, supplying substantially larger conceptual resources yields no additional quantitative benefit. Post-Holocaust antisemitism poses the most persistent challenge across models and configurations. Analysis of explanations further reveals systematic limitations including overproduction of conceptual references, reliance on lexical cues, overconfidence, and difficulties with subtle or justificatory forms of antisemitism. Our findings highlight both the potential and the remaining limitations of conceptually grounded LLMs for antisemitism detection and reasoning.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Hallucination detection and mitigation, one of 17 directions in LLM reasoning. Builds on 21 of its 54 references that are in the map. First posted on arXiv 6 Jul 2026.
Builds on
21 of 54 references.
References · 21
- 2024The Llama 3 Herd of Modelscited by 14,715
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
- 2023ChatGPT Outperforms Crowd-Workers for Text-Annotation Taskscited by 870
- 2021Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigmcited by 606
- 2021Latent Hatred: A Benchmark for Understanding Implicit Hate Speechcited by 217
- 2023Is ChatGPT better than Human Annotators? Potential and Limitations of ChatGPT in Explaining Implicit Hate Speechcited by 166
- 2023Large Language Models Can be Lazy Learners: Analyze Shortcuts in In-Context Learningcited by 69
- 2024Do LLMs Know When to NOT Answer? Investigating Abstention Abilities of Large Language Modelscited by 57
- 2023Probing LLMs for hate speech detection: strengths and vulnerabilitiescited by 52
- 2023"HOT" ChatGPT: The promise of ChatGPT in detecting and discriminating hateful, offensive, and toxic comments on social mediacited by 45
- 2023From Dogwhistles to Bullhorns: Unveiling Coded Rhetoric with Language Modelscited by 26
- 2024Characterizing LLM Abstention Behavior in Science QA with Context Perturbationscited by 26
9 more
- 2024Eureka: Evaluating and Understanding Large Foundation Modelscited by 18
- 2025Lost in Moderation: How Commercial Content Moderation APIs Over- and Under-Moderate Group-Targeted Hate Speech and Linguistic Variationscited by 13
- 2025MoMoE: Mixture of Moderation Experts Framework for AI-Assisted Online Governancecited by 13
- 2025Can Prompting LLMs Unlock Hate Speech Detection across Languages? A Zero-shot and Few-shot Studycited by 7
- 2025A Modular Taxonomy for Hate Speech Definitions and Its Impact on Zero-Shot LLM Classification Performancecited by 7
- 2022Codes, Patterns and Shapes of Contemporary Online Antisemitism and Conspiracy Narratives -- an Annotation Guide and Labeled German-Language Dataset in the Context of COVID-19cited by 6
- 2023How toxic is antisemitism? Potentials and limitations of automated toxicity scoring for antisemitic online contentcited by 4
- 2025Evaluating Large Language Models for Detecting Antisemitismcited by 3
- 2025MAMA-Memeia! Multi-Aspect Multi-Agent Collaboration for Depressive Symptoms Identification in Memescited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026MIRAGE: Auditing Anti-Muslim Bias in Frontier LLMs Across Reasoning, Agentic, and Time-Coupled Conditions
- 2025Mechanistic Interpretability with SAEs: Probing Religion, Violence, and Geography in Large Language Models
- 2025Is LLM an Overconfident Judge? Unveiling the Capabilities of LLMs in Detecting Offensive Language with Annotation Disagreement
- 2024Automating Thematic Analysis: How LLMs Analyse Controversial Topics
- 2026Cultural Awareness is Represented but Not Decoded: Tracing Mythological Knowledge across 18 Open-Source LLMs
- 2026The Grounding Gap: How LLMs Anchor the Meaning of Abstract Concepts Differently from Humans
- 2024LLMs as Models for Analogical Reasoning
- 2026Two Wrongs, No Right: Auditing Social-Desirability Bias in LLM Annotators for Computational Social Science