# Llama Guard 3-1B-INT4: Compact and Efficient Safeguard for Human-AI Conversations

arXiv: https://arxiv.org/abs/2411.17713 · HTML: https://mlsanity.com/paper/2411.17713/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 14 references that are in the map.
- First posted on arXiv 18 Nov 2024.

## Builds on (most cited first, 8 of 8)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2204.05862/) (2022)
- [Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation](https://mlsanity.com/paper/1308.3432/) (2013)
- [Gemma: Open Models Based on Gemini Research and Technology](https://mlsanity.com/paper/2403.08295/) (2024)
- [Quantizing deep convolutional networks for efficient inference: A whitepaper](https://mlsanity.com/paper/1806.08342/) (2018)
- [A White Paper on Neural Network Quantization](https://mlsanity.com/paper/2106.08295/) (2021)
- [ShortGPT: Layers in Large Language Models are More Redundant Than You Expect](https://mlsanity.com/paper/2403.03853/) (2024)
- [LLM-QAT: Data-Free Quantization Aware Training for Large Language Models](https://mlsanity.com/paper/2305.17888/) (2023)

## Built on by (most cited first, 26 of 26)

- [Agentic Web: Weaving the Next Web with AI Agents](https://mlsanity.com/paper/2507.21206/) (2025)
- [ThinkGuard: Deliberative Slow Thinking Leads to Cautious Guardrails](https://mlsanity.com/paper/2502.13458/) (2025)
- [Towards Reasoning Ability of Small Language Models](https://mlsanity.com/paper/2502.11569/) (2025)
- [WebGuard: Building a Generalizable Guardrail for Web Agents](https://mlsanity.com/paper/2507.14293/) (2025)
- [X-Guard: Multilingual Guard Agent for Content Moderation](https://mlsanity.com/paper/2504.08848/) (2025)
- [Distributional AGI Safety](https://mlsanity.com/paper/2512.16856/) (2025)
- [Evolve the Method, Not the Prompts: Evolutionary Synthesis of Jailbreak Attacks on LLMs](https://mlsanity.com/paper/2511.12710/) (2025)
- [LionGuard 2: Building Lightweight, Data-Efficient & Localised Multilingual Content Moderators](https://mlsanity.com/paper/2507.15339/) (2025)
- [A Survey on Collaborating Small and Large Language Models for Performance, Cost-effectiveness, Cloud-edge Privacy, and Trustworthiness](https://mlsanity.com/paper/2510.13890/) (2025)
- [Benchmarking Large Language Models for Cryptanalysis and Side-Channel Vulnerabilities](https://mlsanity.com/paper/2505.24621/) (2025)
- [LPG: Balancing Efficiency and Policy Reasoning in Latent Policy Guardrails](https://mlsanity.com/paper/2605.17329/) (2026)
- [Attention Misses Visual Risk: Risk-Adaptive Steering for Multimodal Safety Alignment](https://mlsanity.com/paper/2510.13698/) (2025)
- [An AI Agent Execution Environment to Safeguard User Data](https://mlsanity.com/paper/2604.19657/) (2026)
- [AISA: Awakening Intrinsic Safety Awareness in Large Language Models against Jailbreak Attacks](https://mlsanity.com/paper/2602.13547/) (2026)
- [MobileMoE: Scaling On-Device Mixture of Experts](https://mlsanity.com/paper/2605.27358/) (2026)
- [MetaBreak: Jailbreaking Online LLM Services via Special Token Manipulation](https://mlsanity.com/paper/2510.10271/) (2025)
- [When Search Goes Wrong: Red-Teaming Web-Augmented Large Language Models](https://mlsanity.com/paper/2510.09689/) (2025)
- [MobileLLM-Flash: Latency-Guided On-Device LLM Design for Industry Scale Deployment](https://mlsanity.com/paper/2603.15954/) (2026)
- [Compliance-Scored Best-of-N Guardrail Orchestration for Multimodal Document Generation in Payments Dispute Defense](https://mlsanity.com/paper/2606.01513/) (2026)
- [Safety Beyond the Interface: Detecting Harm via Latent States in Large Language Models](https://mlsanity.com/paper/2609.19472/) (2026)
- [To Lie or Not to Lie? Investigating The Biased Spread of Global Lies by LLMs](https://mlsanity.com/paper/2604.06552/) (2026)
- [RecurGuard: Runtime Monitoring for Reasoning-Token Consumption Attacks](https://mlsanity.com/paper/2606.07968/) (2026)
- [Distilling Safe LLM Systems via Soft Prompts for On Device Settings](https://mlsanity.com/paper/2606.09388/) (2026)
- [Verifying Intent and Harm: A Unified Defense Against LLM-Generated Threats](https://mlsanity.com/paper/2606.26377/) (2026)
- [Sapien: A Stateful Policy Engine for Autonomous AI Agents](https://mlsanity.com/paper/2610.00797/) (2026)
- [SAFESHIELD: A Decision-Organization Framework for Deployment-Time Safety of Small Language Models](https://mlsanity.com/paper/2610.07276/) (2026)

## Abstract (arXiv)

This paper presents Llama Guard 3-1B-INT4, a compact and efficient Llama Guard model, which has been open-sourced to the community during Meta Connect 2024. We demonstrate that Llama Guard 3-1B-INT4 can be deployed on resource-constrained devices, achieving a throughput of at least 30 tokens per second and a time-to-first-token of 2.5 seconds or less on a commodity Android mobile CPU. Notably, our experiments show that Llama Guard 3-1B-INT4 attains comparable or superior safety moderation scores to its larger counterpart, Llama Guard 3-1B, despite being approximately 7 times smaller in size (440MB).