A Study on Question-Answer Dataset for LLM Safety Evaluation with a Focus on Illegal Activities
Not on the map. Why?
Abstract (arXiv)
In this paper, we discuss question-answer dataset for LLM safety evaluation, with a focus on illegal activities. Specifically, on the basis of manual analysis of AnswerCarefully, we introduce several additional information, methods for creating question-answer examples, and a rubric for evaluating LLM-generated responses. The outcomes of this study are intended to be shared with the "JAI-Trust" project.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 3 of its 8 references that are in the map. First posted on arXiv 28 May 2026.
Builds on
3 of 8 references.
References · 3
- 2024The Llama 3 Herd of Modelscited by 14,715
- 2024A Survey on LLM-as-a-Judgecited by 1,423
- 2019When Choosing Plausible Alternatives, Clever Hans can be Clevercited by 40
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025QGuard:Question-based Zero-shot Guard for Multi-modal LLM Safety
- 2023ToolQA: A Dataset for LLM Question Answering with External Tools
- 2025Evaluating the efficacy of LLM Safety Solutions : The Palit Benchmark Dataset
- 2024SecBench: A Comprehensive Multi-Dimensional Benchmarking Dataset for LLMs in Cybersecurity
- 2025SafeLawBench: Towards Safe Alignment of Large Language Models
- 2023Fake Alignment: Are LLMs Really Aligned Well?
- 2024AttackQA: Development and Adoption of a Dataset for Assisting Cybersecurity Operations using Fine-tuned and Open-Source LLMs
- 2024How (un)ethical are instruction-centric responses of LLMs? Unveiling the vulnerabilities of safety guardrails to harmful queries