mlsanityData to 14 Oct 2026

COGNIT-Guard: Calibrated Standalone Direct-Decision Guardrails with Heterogeneous CPU-NPU Confidence Cascading under Explicit Latency and False-Positive Constraints

arXiv 2609.33671 ↗First posted 27 Sep 2026cs.CR cs.LGcited by 0

On the map: LLM safety and security → Safety alignment, refusal and guardrails

Abstract (arXiv)

When must a foundation-model safety gateway generate tokens, and when should it directly output a calibrated decision? We study calibrated standalone direct-decision foundation models for real-time pre-ingestion safety guardrails, jointly addressing probability calibration, dual-use false-positive control, and heterogeneous CPU-NPU routing under explicit latency SLOs. Pre-ingestion guardrails must screen prompts prior to target-LLM prefill with low false alarms on benign compliance inquiries; however, shallow classifiers are brittle to phrasing shifts, hidden-state probes require coupling to a target LLM, and generative guards incur high decoding latency and dual-use false positives. We present COGNIT-Guard, coupling a validation-calibrated CPU fast gatekeeper with confidence-gated escalation to an NPU-resident 322M bidirectional direct-decision model (Laya-322M) under an asymmetric false-positive penalty. On the clean unseen DUCS-Bench test split (N=607), COGNIT-Guard achieves 98.85% accuracy (McNemar p = 1.19 × 10-4 vs. ML), reduces benign FPR to 0.42% (1/238; Fisher's exact p = 8.23 × 10-4 vs. ML), and attains 1.12% ECE and 0.0104 Brier score. On Huawei Ascend 910C NPUs, pure NPU inference runs in 21.77 ms mean latency (45.90 QPS), while the live serial CPU-NPU cascade (θ^*deploy=0.70) achieves 41.63 ms mean latency (P50: 39.47 ms, 99.23% accuracy, 0.00% FPR). Evaluation on SafetyBench-ZH (N=2,100) and comparison against a bi-encoder direct-decision baseline (CLM-8B) disentangle in-domain gains, OOD alignment tax (60.33% → 56.81% on Laya; 55.10% on domain CLM-8B), and experience replay recovery, restoring OOD accuracy to 64.10%-65.05% and reaching 99.67%-99.84% in-domain accuracy with 0.00%-0.42% FPR.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Safety alignment, refusal and guardrails, one of 12 directions in LLM safety and security. Builds on 8 of its 18 references that are in the map. First posted on arXiv 27 Sep 2026.

Builds on

8 of 18 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024OR-Bench: An Over-Refusal Benchmark for Large Language Models31 May

Other references · 7

  1. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,166
  2. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,207
  3. 2023XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Modelscited by 505
  4. 2023NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Railscited by 365
  5. 2024ShieldGemma: Generative AI Content Moderation Based on Gemmacited by 243
  6. 2023SafetyBench: Evaluating the Safety of Large Language Modelscited by 199
  7. 2026Beyond Over-Refusal: Defending Indirect Prompt Injection via Latent Instruction Manifoldscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026ConsisGuard: Aligning Safety Deliberation with Policy Enforcement in LLM Guardrails
  2. 2026A Dual-Hypothesis Reasoning Framework for LLM Guardrails
  3. 2025Building a Foundational Guardrail for General Agentic Systems via Synthetic Data
  4. 2026AdaGuard: Enhancing Safety and Policy Compliance with Reasoning-Enabled LLM-As-A-Judge Guardrails
  5. 2026Test-Time Training Undermines Safety Guardrails
  6. 2026Robust and Efficient Guardrails with Latent Reasoning
  7. 2026NEXUS: Structured Runtime Safety for Tool-Using LLM Agents
  8. 2026Triaging Threats to Specialized Guardrails