mlsanityData to 14 Oct 2026

Can't say cant? Measuring and Reasoning of Dark Jargons in Large Language Models

arXiv 2405.00718 ↗First posted 25 Apr 2024cs.CL cs.AIcited by 0

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Ensuring the resilience of Large Language Models (LLMs) against malicious exploitation is paramount, with recent focus on mitigating offensive responses. Yet, the understanding of cant or dark jargon remains unexplored. This paper introduces a domain-specific Cant dataset and CantCounter evaluation framework, employing Fine-Tuning, Co-Tuning, Data-Diffusion, and Data-Analysis stages. Experiments reveal LLMs, including ChatGPT, are susceptible to cant bypassing filters, with varying recognition accuracy influenced by question types, setups, and prompt clues. Updated models exhibit higher acceptance rates for cant queries. Moreover, LLM reactions differ across domains, e.g., reluctance to engage in racism versus LGBT topics. These findings underscore LLMs' understanding of cant and reflect training data characteristics and vendor approaches to sensitive topics. Additionally, we assess LLMs' ability to demonstrate reasoning capabilities. Access to our datasets and code is available at https://github.com/cistineup/CantCounter.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 6 of its 44 references that are in the map. First posted on arXiv 25 Apr 2024.

Builds on

6 of 44 references.

References · 6

  1. 2021A Dataset of Information-Seeking Questions and Answers Anchored in Research Paperscited by 486
  2. 2023GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Ciphercited by 423
  3. 2023Multi-step Jailbreaking Privacy Attacks on ChatGPTcited by 333
  4. 2023Can ChatGPT replace StackOverflow? A Study on Robustness and Reliability of Large Language Model Code Generationcited by 73
  5. 2023Adversarial Demonstration Attacks on Large Language Modelscited by 63
  6. 2023Generating Phishing Attacks using ChatGPTcited by 16

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Benchmarking Gaslighting Negation Attacks Against Multimodal Large Language Models
  2. 2023Probing LLMs for hate speech detection: strengths and vulnerabilities
  3. 2025Prompt Injection as an Emerging Threat: Evaluating the Resilience of Large Language Models
  4. 2025LLMs are Vulnerable to Malicious Prompts Disguised as Scientific Language
  5. 2026IndicSafeEval: Safety Robustness of Large Language Models under Multilingual Persuasive Jailbreak Attacks
  6. 2026No One Model Catches Every Harm: Benchmarking Content Moderation Across Safety Scenarios
  7. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity
  8. 2025Evaluating Large Language Models for Detecting Antisemitism