mlsanityData to 14 Oct 2026

Red Teaming Contemporary AI Models: Insights from Spanish and Basque Perspectives

arXiv 2503.10192 ↗First posted 13 Mar 2025cs.SE cs.CLcited by 6

On the map: LLM safety and security → Prompt injection and agent security

Abstract (arXiv)

The battle for AI leadership is on, with OpenAI in the United States and DeepSeek in China as key contenders. In response to these global trends, the Spanish government has proposed ALIA, a public and transparent AI infrastructure incorporating small language models designed to support Spanish and co-official languages such as Basque. This paper presents the results of Red Teaming sessions, where ten participants applied their expertise and creativity to manually test three of the latest models from these initiativesunicodex2013OpenAI o3-mini, DeepSeek R1, and ALIA Salamandraunicodex2013focusing on biases and safety concerns. The results, based on 670 conversations, revealed vulnerabilities in all the models under test, with biased or unsafe responses ranging from 29.5% in o3-mini to 50.6% in Salamandra. These findings underscore the persistent challenges in developing reliable and trustworthy AI systems, particularly those intended to support Spanish and Basque languages.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Placed in Prompt injection and agent security, one of 12 directions in LLM safety and security. Builds on 14 of its 24 references that are in the map. First posted on arXiv 13 Mar 2025.

Builds on

14 of 24 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025OpenAI's Approach to External Red Teaming for AI Models and Systems24 Jan
  2. 2024Red-Teaming for Generative AI: Silver Bullet or Security Theater?29 Jan
  3. 2023MART: Improving LLM Safety with Multi-round Automatic Red-Teaming13 Nov
  4. 2023Summon a Demon and Bind it: A Grounded Theory of LLM Red Teaming10 Nov
  5. 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned23 Aug
  6. 2022Red Teaming Language Models with Language Models7 Feb

Other references · 8

  1. 2017Attention Is All You Needcited by 60,361
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2024The Llama 3 Herd of Modelscited by 14,715
  4. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  5. 2024SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusalcited by 240
  6. 2023FLIRT: Feedback Loop In-context Red Teamingcited by 83
  7. 2024S-Eval: Towards Automated and Comprehensive Safety Evaluation for Large Language Modelscited by 33
  8. 2025ASTRAL: Automated Safety Testing of Large Language Modelscited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 5; 2026: 1.

6 papers.

  1. 2025Safety in Large Reasoning Models: A Surveycited by 64
  2. 2025A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Modelscited by 25
  3. 2025Beyond Safe Answers: A Benchmark for Evaluating True Risk Awareness in Large Reasoning Modelscited by 7
  4. 2026Towards a Mechanistic Understanding of Large Reasoning Models: A Survey of Training, Inference, and Failurescited by 6
  5. 2025Red Teaming AI Red Teamingcited by 3
  6. 2025Sentra-Guard: A Real-Time Multilingual Defense Against Adversarial LLM Promptscited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2025Building Safe GenAI Applications: An End-to-End Overview of Red Teaming for Large Language Models
  2. 2025CoP: Agentic Red-teaming for Large Language Models using Composition of Principles
  3. 2024Automated Red Teaming with GOAT: the Generative Offensive Agent Tester
  4. 2026OpenRT: An Open-Source Red Teaming Framework for Multimodal LLMs
  5. 2025Multi-lingual Multi-turn Automated Red Teaming for LLMs
  6. 2026Uncovering Linguistic Fragility in Vision-Language-Action Models via Diversity-Aware Red Teaming
  7. 2025AIRTBench: Measuring Autonomous AI Red Teaming Capabilities in Language Models
  8. 2024Towards Red Teaming in Multimodal and Multilingual Translation