# AnswerCarefully: A Dataset for Improving the Safety of Japanese LLM Output

arXiv: https://arxiv.org/abs/2506.02372 · HTML: https://mlsanity.com/paper/2506.02372/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 12 references that are in the map.
- First posted on arXiv 3 Jun 2025.

## Builds on (most cited first, 4 of 4)

- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversations](https://mlsanity.com/paper/2312.06674/) (2023)
- [A Chinese Dataset for Evaluating the Safeguards in Large Language Models](https://mlsanity.com/paper/2402.12193/) (2024)
- [JBBQ: Japanese Bias Benchmark for Analyzing Social Biases in Large Language Models](https://mlsanity.com/paper/2406.02050/) (2024)

## Built on by (most cited first, 4 of 4)

- [The Scales of Justitia: A Comprehensive Survey on Safety Evaluation of LLMs](https://mlsanity.com/paper/2506.11094/) (2025)
- [JMedEthicBench: A Multi-Turn Conversational Benchmark for Evaluating Medical Safety in Japanese Large Language Models](https://mlsanity.com/paper/2601.01627/) (2026)
- [Enhancing Japanese Large Language Models with Reasoning Vectors](https://mlsanity.com/paper/2508.02913/) (2025)
- [AdaCultureSafe: Adaptive Cultural Safety Grounded by Cultural Knowledge in Large Language Models](https://mlsanity.com/paper/2603.08275/) (2026)

## Abstract (arXiv)

In this paper we present AnswerCarefully, a dataset for promoting the safety and appropriateness of Japanese LLM outputs. The dataset consists of 1,800 pairs of questions and reference answers, where the questions require special attention in answering. It covers a wide range of risk categories established in prior English-language datasets, but the data samples are original in that they are manually created to reflect the socio-cultural context of LLM usage in Japan. We show that using this dataset for instruction to fine-tune a Japanese LLM led to improved output safety without compromising the utility of general responses. We also report the results of a safety evaluation of 12 Japanese LLMs using this dataset as a benchmark. Finally, we describe the latest update on the dataset which provides English translations and annotations of the questions, aimed at facilitating the derivation of similar datasets in different languages and regions.