# Comprehensive Evaluation of ChatGPT Reliability Through Multilingual Inquiries

arXiv: https://arxiv.org/abs/2312.10524 · HTML: https://mlsanity.com/paper/2312.10524/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Placed in Jailbreaking, one of 12 directions in LLM safety and security.
- Builds on 16 of its 37 references that are in the map.
- First posted on arXiv 16 Dec 2023.

## Placed in

- LLM safety and security / [Jailbreaking](https://mlsanity.com/safety/jailbreaking/)

## Builds on (most cited first, 16 of 16)

- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Red Teaming Language Models with Language Models](https://mlsanity.com/paper/2202.03286/) (2022)
- [A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity](https://mlsanity.com/paper/2302.04023/) (2023)
- [Prompt Injection attack against LLM-integrated Applications](https://mlsanity.com/paper/2306.05499/) (2023)
- [GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Prompts](https://mlsanity.com/paper/2309.10253/) (2023)
- [Jailbreaking ChatGPT via Prompt Engineering: An Empirical Study](https://mlsanity.com/paper/2305.13860/) (2023)
- [Multilingual Jailbreak Challenges in Large Language Models](https://mlsanity.com/paper/2310.06474/) (2023)
- [ChatGPT Beyond English: Towards a Comprehensive Evaluation of Large Language Models in Multilingual Learning](https://mlsanity.com/paper/2304.05613/) (2023)
- [Is ChatGPT A Good Translator? Yes With GPT-4 As The Engine](https://mlsanity.com/paper/2301.08745/) (2023)
- [Language Contamination Helps Explain the Cross-lingual Capabilities of English Pretrained Models](https://mlsanity.com/paper/2204.08110/) (2022)
- [DLFuzz: Differential Fuzzing Testing of Deep Learning Systems](https://mlsanity.com/paper/1808.09413/) (2018)
- [Evaluating Fuzz Testing](https://mlsanity.com/paper/1808.09700/) (2018)
- [RobOT: Robustness-Oriented Testing for Deep Learning Systems](https://mlsanity.com/paper/2102.05913/) (2021)
- [Safeguarding Crowdsourcing Surveys from ChatGPT with Prompt Injection](https://mlsanity.com/paper/2306.08833/) (2023)
- [On the Relationship Between the Developer's Perceptible Race and Ethnicity and the Evaluation of Contributions in OSS](https://mlsanity.com/paper/2104.06143/) (2021)
- [Women's Participation in Open Source Software: A Survey of the Literature](https://mlsanity.com/paper/2105.08777/) (2021)

## Built on by (most cited first, 11 of 11)

- [Red-Teaming for Generative AI: Silver Bullet or Security Theater?](https://mlsanity.com/paper/2401.15897/) (2024)
- [Multilingual Large Language Model: A Survey of Resources, Taxonomy and Frontiers](https://mlsanity.com/paper/2404.04925/) (2024)
- [RTP-LX: Can LLMs Evaluate Toxicity in Multilingual Scenarios?](https://mlsanity.com/paper/2404.14397/) (2024)
- [Sandwich attack: Multi-language Mixture Adaptive Attack on LLMs](https://mlsanity.com/paper/2404.07242/) (2024)
- [Cognitive Overload Attack:Prompt Injection for Long Context](https://mlsanity.com/paper/2410.11272/) (2024)
- [A Chinese Dataset for Evaluating the Safeguards in Large Language Models](https://mlsanity.com/paper/2402.12193/) (2024)
- [Evaluating and Mitigating Linguistic Discrimination in Large Language Models](https://mlsanity.com/paper/2404.18534/) (2024)
- [Semantic Aware Linear Transfer by Recycling Pre-trained Language Models for Cross-lingual Transfer](https://mlsanity.com/paper/2505.10945/) (2025)
- [Improving LLM Outputs Against Jailbreak Attacks with Expert Model Integration](https://mlsanity.com/paper/2505.17066/) (2025)
- [The "Knowledge-Behavior Gap" in Cultural Taboo Safety of Large Language Models](https://mlsanity.com/paper/2608.12341/) (2026)
- [Language Chain in Alignment: Cross-lingual Ranking Preference Optimization](https://mlsanity.com/paper/2608.23149/) (2026)

## Abstract (arXiv)

ChatGPT is currently the most popular large language model (LLM), with over 100 million users, making a significant impact on people's lives. However, due to the presence of jailbreak vulnerabilities, ChatGPT might have negative effects on people's lives, potentially even facilitating criminal activities. Testing whether ChatGPT can cause jailbreak is crucial because it can enhance ChatGPT's security, reliability, and social responsibility. Inspired by previous research revealing the varied performance of LLMs in different language translations, we suspected that wrapping prompts in multiple languages might lead to ChatGPT jailbreak. To investigate this, we designed a study with a fuzzing testing approach to analyzing ChatGPT's cross-linguistic proficiency. Our study includes three strategies by automatically posing different formats of malicious questions to ChatGPT: (1) each malicious question involving only one language, (2) multilingual malicious questions, (3) specifying that ChatGPT responds in a language different from the prompts. In addition, we also combine our strategies by utilizing prompt injection templates to wrap the three aforementioned types of questions. We examined a total of 7,892 Q&A data points, discovering that multilingual wrapping can indeed lead to ChatGPT's jailbreak, with different wrapping methods having varying effects on jailbreak probability. Prompt injection can amplify the probability of jailbreak caused by multilingual wrapping. This work provides insights for OpenAI developers to enhance ChatGPT's support for language diversity and inclusion.