Beyond Context: Large Language Models' Failure to Grasp Users' Intent
On the map: LLM safety and security → Safety alignment, refusal and guardrails
Abstract (arXiv)
Current Large Language Models (LLMs) safety approaches focus on explicitly harmful content while overlooking a critical vulnerability: the inability to understand context and recognize user intent. This creates exploitable vulnerabilities that malicious users can systematically leverage to circumvent safety mechanisms. We empirically evaluate multiple state-of-the-art LLMs, including ChatGPT, Claude, Gemini, and DeepSeek. Our analysis demonstrates the circumvention of reliable safety mechanisms through emotional framing, progressive revelation, and academic justification techniques. Notably, reasoning-enabled configurations amplified rather than mitigated the effectiveness of exploitation, increasing factual precision while failing to interrogate the underlying intent. The exception was Claude Opus 4.1, which prioritized intent detection over information provision in some use cases. This pattern reveals that current architectural designs create systematic vulnerabilities. These limitations require paradigmatic shifts toward contextual understanding and intent recognition as core safety capabilities rather than post-hoc protective mechanisms.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Safety alignment, refusal and guardrails, one of 12 directions in LLM safety and security. Builds on 83 of its 116 references that are in the map. First posted on arXiv 24 Dec 2025.
Builds on
83 of 116 references.
Closest ancestors · 1
Same direction, newest first
- 2025Is Safety Standard Same for Everyone? User-Specific Safety Evaluation of Large Language Models20 Feb
Other references · 82
- 2017Attention Is All You Needcited by 59,932
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2023GPT-4 Technical Reportcited by 18,132
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
- 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
- 2017Graph Attention Networkscited by 8,854
- 2014Explaining and Harnessing Adversarial Examplescited by 7,287
- 2020Scaling Laws for Neural Language Modelscited by 6,411
- 2013Intriguing properties of neural networkscited by 5,114
- 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
38 more
- 2017Deep reinforcement learning from human preferencescited by 3,846
- 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
- 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
- 2020Longformer: The Long-Document Transformercited by 3,362
- 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
- 2022Training Compute-Optimal Large Language Modelscited by 2,927
- 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
- 2018Deep contextualized word representationscited by 2,285
- 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,219
- 2020Extracting Training Data from Large Language Modelscited by 2,144
- 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
- 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
- 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,830
- 2020Big Bird: Transformers for Longer Sequencescited by 1,673
- 2016Concrete Problems in AI Safetycited by 1,582
- 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
- 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
- 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
- 2019VisualBERT: A Simple and Performant Baseline for Vision and Languagecited by 1,291
- 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,235
- 2017Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPRcited by 1,086
- 2019BERT Rediscovers the Classical NLP Pipelinecited by 1,046
- 2019What Does BERT Look At? An Analysis of BERT's Attentioncited by 994
- 2022Red Teaming Language Models with Language Modelscited by 983
- 2020Beyond Accuracy: Behavioral Testing of NLP models with CheckListcited by 979
- 2021Ethical and social risks of harm from Language Modelscited by 957
- 2019Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inferencecited by 864
- 2020A Primer in BERTology: What we know about how BERT workscited by 807
- 2024Evaluating Very Long-Term Conversational Memory of LLM Agentscited by 800
- 2019Adversarial NLI: A New Benchmark for Natural Language Understandingcited by 798
- 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 760
- 2023Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedbackcited by 743
- 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Modelscited by 738
- 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 709
- 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learnedcited by 702
- 2019Compressive Transformers for Long-Range Sequence Modellingcited by 699
- 2017Evasion Attacks against Machine Learning at Test Timecited by 686
- 2019Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailmentcited by 663
50 most cited of 82 All 82 (JSON)
Built on by
Citing papers per year: 2026: 1.
1 paper.