# Building a Domain-specific Guardrail Model in Production

arXiv: https://arxiv.org/abs/2408.01452 · HTML: https://mlsanity.com/paper/2408.01452/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in Safety alignment, refusal and guardrails, one of 12 directions in LLM safety and security.
- Builds on 23 of its 41 references that are in the map.
- First posted on arXiv 24 Jul 2024.

## Placed in

- LLM safety and security / [Safety alignment, refusal and guardrails](https://mlsanity.com/safety/safety-alignment-refusal-and-guardrails/)

## Builds on (most cited first, 23 of 23)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classification](https://mlsanity.com/paper/1903.04561/) (2019)
- [Ex Machina: Personal Attacks Seen at Scale](https://mlsanity.com/paper/1610.08914/) (2016)
- [GPTs are GPTs: An Early Look at the Labor Market Impact Potential of Large Language Models](https://mlsanity.com/paper/2303.10130/) (2023)
- [Recipes for Safety in Open-domain Chatbots](https://mlsanity.com/paper/2010.07079/) (2020)
- [Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims](https://mlsanity.com/paper/2004.07213/) (2020)
- [A Survey of Safety and Trustworthiness of Large Language Models through the Lens of Verification and Validation](https://mlsanity.com/paper/2305.11391/) (2023)
- [Privacy Preserving Machine Learning: Threats and Solutions](https://mlsanity.com/paper/1804.11238/) (2018)
- [Consistency Regularization for Certified Robustness of Smoothed Classifiers](https://mlsanity.com/paper/2006.04062/) (2020)
- [On the Safety of Conversational Models: Taxonomy, Dataset, and Benchmark](https://mlsanity.com/paper/2110.08466/) (2021)
- [Building Guardrails for Large Language Models](https://mlsanity.com/paper/2402.01822/) (2024)
- [Current state of LLM Risks and AI Guardrails](https://mlsanity.com/paper/2406.12934/) (2024)
- [Measuring Conversational Uptake: A Case Study on Student-Teacher Interactions](https://mlsanity.com/paper/2106.03873/) (2021)
- [The Ethics of Interaction: Mitigating Security Threats in LLMs](https://mlsanity.com/paper/2401.12273/) (2024)
- [Exploring the Capabilities and Limitations of Large Language Models in the Electric Energy Sector](https://mlsanity.com/paper/2403.09125/) (2024)
- [How Useful are Educational Questions Generated by Large Language Models?](https://mlsanity.com/paper/2304.06638/) (2023)
- [Certified Robustness for Large Language Models with Self-Denoising](https://mlsanity.com/paper/2307.07171/) (2023)
- [Scalable Educational Question Generation with Pre-trained Language Models](https://mlsanity.com/paper/2305.07871/) (2023)
- [Towards Safer Generative Language Models: A Survey on Safety Risks, Evaluations, and Improvements](https://mlsanity.com/paper/2302.09270/) (2023)
- [GUARD-D-LLM: An LLM-Based Risk Assessment Engine for the Downstream uses of LLMs](https://mlsanity.com/paper/2406.11851/) (2024)
- [ChatGPT-3.5, ChatGPT-4, Google Bard, and Microsoft Bing to Improve Health Literacy and Communication in Pediatric Populations and Beyond](https://mlsanity.com/paper/2311.10075/) (2023)
- [Empowering ChatGPT-Like Large-Scale Language Models with Local Knowledge Base for Industrial Prognostics and Health Management](https://mlsanity.com/paper/2312.14945/) (2023)
- [Guardrails for avoiding harmful medical product recommendations and off-label promotion in generative AI models](https://mlsanity.com/paper/2406.16455/) (2024)

## Built on by (most cited first, 6 of 6)

- [SciSafeEval: A Comprehensive Benchmark for Safety Alignment of Large Language Models in Scientific Tasks](https://mlsanity.com/paper/2410.03769/) (2024)
- [PROMPTEVALS: A Dataset of Assertions and Guardrails for Custom Production Large Language Model Pipelines](https://mlsanity.com/paper/2504.14738/) (2025)
- [LPG: Balancing Efficiency and Policy Reasoning in Latent Policy Guardrails](https://mlsanity.com/paper/2605.17329/) (2026)
- [A Framework for Situating Innovations, Opportunities, and Challenges in Advancing Vertical Systems with Large AI Models](https://mlsanity.com/paper/2504.02793/) (2025)
- [Bridging Today and the Future of Humanity: AI Safety in 2024 and Beyond](https://mlsanity.com/paper/2410.18114/) (2024)
- [A Grounded Observer Framework for Establishing Guardrails for Foundation Models in Socially Sensitive Domains](https://mlsanity.com/paper/2412.18639/) (2024)

## Abstract (arXiv)

Generative AI holds the promise of enabling a range of sought-after capabilities and revolutionizing workflows in various consumer and enterprise verticals. However, putting a model in production involves much more than just generating an output. It involves ensuring the model is reliable, safe, performant and also adheres to the policy of operation in a particular domain. Guardrails as a necessity for models has evolved around the need to enforce appropriate behavior of models, especially when they are in production. In this paper, we use education as a use case, given its stringent requirements of the appropriateness of content in the domain, to demonstrate how a guardrail model can be trained and deployed in production. Specifically, we describe our experience in building a production-grade guardrail model for a K-12 educational platform. We begin by formulating the requirements for deployment to this sensitive domain. We then describe the training and benchmarking of our domain-specific guardrail model, which outperforms competing open- and closed- instruction-tuned models of similar and larger size, on proprietary education-related benchmarks and public benchmarks related to general aspects of safety. Finally, we detail the choices we made on architecture and the optimizations for deploying this service in production; these range across the stack from the hardware infrastructure to the serving layer to language model inference optimizations. We hope this paper will be instructive to other practitioners looking to create production-grade domain-specific services based on generative AI and large language models.