mlsanityData to 14 Oct 2026

Do LLMs Hold Their Values? MANTA: A Multi-Turn Adversarial Benchmark for Animal Welfare Reasoning

arXiv 2605.16301 ↗First posted 18 Apr 2026cs.CY cs.AI cs.LGcited by 1

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Evaluating animal welfare reasoning in LLMs remains an open challenge despite rapid deployment in consumer and professional contexts where welfare considerations appear implicitly in everyday queries. Existing benchmarks such as AnimalHarmBench evaluate this through single-turn, explicitly framed questions, measuring whether models avoid harmful content when directly asked. This approach overlooks two failure modes: alignment degradation under sustained adversarial pressure, and moral sensitivity (whether a model spontaneously surfaces welfare stakes in everyday queries). To fill this gap, we construct MANTA, a benchmark of 1,088 five-turn conversations progressing from an implicit Turn-1 scenario through an explicit welfare prompt to three adversarial pressure rounds drawn from a five-type taxonomy: Social, Cultural, Economic, Pragmatic, and Epistemic. We score conversations on two dimensions: Animal Welfare Value Stability (AWVS, primary) and Animal Welfare Moral Sensitivity (AWMS, diagnostic). We evaluate seven frontier models: Claude Opus 4.7, GPT-5.5, DeepSeek V4, Llama 3.3 70B, Mistral Small, Grok 4.3, and Gemini 3.1 Flash Lite. Multi-turn evaluation captures behavior single-turn benchmarks miss: 4 of 7 models change rank relative to Turn 1 moral-sensitivity scores", including Gemini Flash Lite, which drops from fifth on AWMS to last on AWVS. AWMS and AWVS are positively but imperfectly correlated, suggesting moral-sensitivity tests capture a stable but incomplete component of model behavior under pressure. MANTA also enables a species-by-pressure interaction matrix unavailable to prior benchmarks, showing welfare robustness depends jointly on the animal and pressure applied; companion animals score above wild animals, which score above farmed animals and invertebrates. We release the dataset, scripted pressure plans, judge prompts, and analysis code.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 25 of its 65 references that are in the map. First posted on arXiv 18 Apr 2026.

Builds on

25 of 65 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025EvalMORAAL: Interpretable Chain-of-Thought and LLM-as-Judge Evaluation for Moral Alignment in Large Language Models7 Oct

Other references · 24

  1. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  2. 2024A Survey on LLM-as-a-Judgecited by 1,423
  3. 2023Towards Understanding Sycophancy in Language Modelscited by 1,040
  4. 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 836
  5. 2023Prometheus: Inducing Fine-grained Evaluation Capability in Language Modelscited by 575
  6. 2025LLMs Get Lost In Multi-Turn Conversationcited by 429
  7. 2024Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attackcited by 390
  8. 2024Alignment faking in large language modelscited by 360
  9. 2023MINT: Evaluating LLMs in Multi-turn Interaction with Tools and Language Feedbackcited by 297
  10. 2024Self-Preference Bias in LLM-as-a-Judgecited by 258
  11. 2025MultiChallenge: A Realistic Multi-Turn Conversation Evaluation Benchmark Challenging to Frontier LLMscited by 180
  12. 2025Large Language Models Often Know When They Are Being Evaluatedcited by 128
12 more
  1. 2025Measuring what Matters: Construct Validity in Large Language Model Benchmarkscited by 119
  2. 2023Cooperation, Competition, and Maliciousness: LLM-Stakeholders Interactive Negotiationcited by 96
  3. 2024Clio: Privacy-Preserving Insights into Real-World AI Usecited by 72
  4. 2024Automated Red Teaming with GOAT: the Generative Offensive Agent Testercited by 45
  5. 2024FairMT-Bench: Benchmarking Fairness for Multi-turn Dialogue in Conversational LLMscited by 36
  6. 2024Beyond correlation: The Impact of Human Uncertainty in Measuring the Effectiveness of Automatic Evaluation and LLM-as-a-Judgecited by 31
  7. 2025Know Thy Judge: On the Robustness Meta-Evaluation of LLM Safety Judgescited by 24
  8. 2025PropensityBench: Evaluating Latent Safety Risks in Large Language Models via an Agentic Approachcited by 22
  9. 2025What do Large Language Models Say About Animals? Investigating Risks of Animal Harm in Generated Textcited by 9
  10. 2025AutoAdv: Automated Adversarial Prompting for Multi-Turn Jailbreaking of Large Language Modelscited by 8
  11. 2024The Case for Animal-Friendly AIcited by 4
  12. 2026Alignment midtraining for animalscited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Your AI Travel Agent Would Book You a Bullfight: An Agentic Benchmark for Implicit Animal Welfare in Frontier AI Modelscited by 2

Similar, not cited

Close in wording, no citation either way

  1. 2026Assert, don't describe: Linguistic features that shift LLM reasoning about animal welfare
  2. 2026Normative Robustness as a Frontier for Non-Verifiable Reasoning in LLMs
  3. 2025MORABLES: A Benchmark for Assessing Abstract Moral Reasoning in LLMs with Fables
  4. 2025Survey-to-Behavior: Downstream Alignment of Human Values in LLMs via Survey Questions
  5. 2026Unsupervised Elicitation of Moral Values from Language Models
  6. 2025Structured Moral Reasoning in Language Models: A Value-Grounded Evaluation Framework
  7. 2025Do LLMs Align Human Values Regarding Social Biases? Judging and Explaining Social Biases with LLMs
  8. 2025Normative Conflicts and Shallow AI Alignment