mlsanityData to 7 Oct 2026

Semalith v1.4: A Calibrated 184M Safety Classifier Achieving State-of-the-Art Prompt-Injection Detection at 44x Fewer Parameters than Llama-Guard-3-8B

arXiv 2607.22545 ↗First posted 6 May 2026cs.LG cs.AI cs.CLcited by 0

On the map: LLM agents and tool use → Agent threats, guardrails and governance

also in 1 more
Abstract (arXiv)

Deploying large language models in financial-services and agentic settings requires safety classifiers that simultaneously handle prompt injection, regulatory compliance, and general harm, a combination no existing open guardrail addresses in a single inference pass. Semalith v1.4 is a 184M-parameter DeBERTa-v3-base classifier performing simultaneous three-axis safety classification including prompt injection, general harm, and financial-services regulatory compliance, in a single forward pass. Its 22-class head (BENIGN, nine prompt-injection sub-types, general-harm, eleven BFSI labels) is trained with a 4-class auxiliary super-category head under jointly weighted loss, on a 76,204-row corpus mined from 49 public sources with SHA-1 deduplication against every held-out evaluation set, with 21 of 22 benchmarks at zero contamination (max 0.22%). Against Llama-Guard-3-8B on 22 held-out benchmarks, Semalith v1.4 wins every prompt-injection evaluation (7/7) and 11 of 18 benchmarks overall at 44x fewer parameters, with FPR = 0.000 on 208 benign agentic prompts vs 0.063 for Llama-Guard-3-8B. On general-harm benchmarks (WildGuardMix, HEx-PHI, HarmBench), Llama-Guard-3 leads; this complementary split is documented in Section 4. Six measured weak spots are disclosed in Section 6. Deployment guidance: v1.3 is recommended for conversational moderation deployments (ToxicChat F1 0.624); v1.4 is recommended when BFSI label coverage or zero-FPR on benign agentic prompts is the priority.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 15 of its 33 references that are in the map. First posted on arXiv 6 May 2026.

Builds on

15 of 33 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents11 Oct

Other references · 14

  1. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  2. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
  3. 2021DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharingcited by 1,219
  4. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,124
  5. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,106
  6. 2023BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Datasetcited by 828
  7. 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 760
  8. 2024The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearningcited by 488
  9. 2024WildGuard: Open One-Stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMscited by 460
  10. 2024SALAD-Bench: A Hierarchical and Comprehensive Safety Benchmark for Large Language Modelscited by 225
  11. 2023ToxicChat: Unveiling Hidden Challenges of Toxicity Detection in Real-World User-AI Conversationcited by 224
  12. 2023SimpleSafetyTests: a Test Suite for Identifying Critical Safety Risks in Large Language Modelscited by 72
2 more
  1. 2023AART: AI-Assisted Red-Teaming with Diverse Data Generation for New LLM-powered Applicationscited by 53
  2. 2024Granite Guardiancited by 20

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026HaloGuard 1.0: An Open Weights Constitutional Classifier for Multilingual AI Safety
  2. 2026Prompt-Induced Score Variance in Zero-Shot Binary Vision-Language Safety Classification
  3. 2025OMNIGUARD: An Efficient Approach for AI Safety Moderation Across Languages and Modalities
  4. 2026SalamahBench: Toward Standardized Safety Evaluation for Arabic Language Models
  5. 2025SafeTuneBed: A Toolkit for Benchmarking LLM Safety Alignment in Fine-Tuning
  6. 2026CourtGuard: A Model-Agnostic Framework for Zero-Shot Policy Adaptation in LLM Safety
  7. 2025aiXamine: Simplified LLM Safety and Security
  8. 2026Reflect-Guard: Enhancing LLM Safeguards against Adversarial Prompts via Logical Self-Reflection