mlsanityData to 7 Oct 2026

Shh, don't say that! Domain Certification in LLMs

arXiv 2502.19320 ↗First posted 26 Feb 2025cs.CL cs.AI cs.CRcited by 7

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) are often deployed to perform constrained tasks, with narrow domains. For example, customer support bots can be built on top of LLMs, relying on their broad language understanding and capabilities to enhance performance. However, these LLMs are adversarially susceptible, potentially generating outputs outside the intended domain. To formalize, assess, and mitigate this risk, we introduce domain certification; a guarantee that accurately characterizes the out-of-domain behavior of language models. We then propose a simple yet effective approach, which we call VALID that provides adversarial bounds as a certificate. Finally, we evaluate our method across a diverse set of datasets, demonstrating that it yields meaningful certificates, which bound the probability of out-of-domain samples tightly with minimum penalty to refusal behavior.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 58 of its 81 references that are in the map. First posted on arXiv 26 Feb 2025.

Builds on

58 of 81 references.

References · 58

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  5. 2020Scaling Laws for Neural Language Modelscited by 6,411
  6. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  7. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  8. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  9. 2017Deep reinforcement learning from human preferencescited by 3,846
  10. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  11. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  12. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
38 more
  1. 2020Supervised Contrastive Learningcited by 2,803
  2. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
  3. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
  4. 2020Energy-based Out-of-distribution Detectioncited by 1,296
  5. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,271
  6. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,255
  7. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,124
  8. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,106
  9. 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,023
  10. 2022Red Teaming Language Models with Language Modelscited by 983
  11. 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 760
  12. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Modelscited by 738
  13. 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 709
  14. 2023Safe RLHF: Safe Reinforcement Learning from Human Feedbackcited by 630
  15. 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automaticallycited by 619
  16. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 608
  17. 2022Contrastive Decoding: Open-ended Text Generation as Optimizationcited by 598
  18. 2017Adversarial Examples for Evaluating Reading Comprehension Systemscited by 568
  19. 2020Scaling Laws for Autoregressive Generative Modelingcited by 510
  20. 2019Likelihood Ratios for Out-of-Distribution Detectioncited by 451
  21. 2023Are aligned neural networks adversarially aligned?cited by 305
  22. 2023A Survey of Reinforcement Learning from Human Feedbackcited by 259
  23. 2023Certifying LLM Safety against Adversarial Promptingcited by 248
  24. 2024Rainbow Teaming: Open-Ended Generation of Diverse Adversarial Promptscited by 189
  25. 2023Automatically Auditing Large Language Models via Discrete Optimizationcited by 187
  26. 2023Grammar-Constrained Decoding for Structured NLP Tasks without Finetuningcited by 186
  27. 2024Generalized Preference Optimization: A Unified Approach to Offline Alignmentcited by 155
  28. 2023Few-shot Fine-tuning vs. In-context Learning: A Fair Comparison and Evaluationcited by 134
  29. 2022Revisiting Neural Scaling Laws in Language and Visioncited by 129
  30. 2021Advances in adversarial attacks and defenses in computer vision: A surveycited by 127
  31. 2024Improved Techniques for Optimization-Based Jailbreaking on Large Language Modelscited by 98
  32. 2024Safeguarding Large Language Models: A Surveycited by 73
  33. 2021Contrastive Out-of-Distribution Detection for Pretrained Transformerscited by 70
  34. 2021Revisiting Mahalanobis Distance for Transformer-Based Out-of-Domain Detectioncited by 67
  35. 2021How Should Pre-Trained Language Models Be Fine-Tuned Towards Adversarial Robustness?cited by 49
  36. 2019Likelihood Ratios and Generative Classifiers for Unsupervised Out-of-Domain Detection In Task Oriented Dialogcited by 33
  37. 2023Prompt Packer: Deceiving LLMs through Compositional Instruction with Hidden Attackscited by 30
  38. 2023Is Fine-tuning Needed? Pre-trained Language Models Are Near Perfect for Out-of-Domain Detectioncited by 29

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 6; 2026: 1.

7 papers.

  1. 2025CARES: Comprehensive Evaluation of Safety and Adversarial Robustness in Medical LLMscited by 26
  2. 2025Lifelong Safety Alignment for Language Modelscited by 13
  3. 2025MIP against Agent: Malicious Image Patches Hijacking Multimodal OS Agentscited by 12
  4. 2025Agents Require Metacognitive and Strategic Reasoning to Succeed in the Coming Labor Marketscited by 2
  5. 2025How Catastrophic is Your LLM? Certifying Risk in Conversationcited by 2
  6. 2025BEAVER: An Efficient Deterministic LLM Verifiercited by 2
  7. 2026Mixture-of-Expert Blocks Contain Strong Hallucination Detection Signalscited by 0