mlsanityData to 7 Oct 2026

Less Diverse, Less Safe: The Indirect But Pervasive Risk of Test-Time Scaling in Large Language Models

arXiv 2510.08592 ↗First posted 4 Oct 2025cs.CL cs.AI cs.LGcited by 2

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Test-Time Scaling (TTS) improves LLM reasoning by exploring multiple candidate responses and then operating over this set to find the best output. A tacit premise behind TTS is that sufficiently diverse candidate pools enhance reliability. In this work, we show that this assumption in TTS introduces a previously unrecognized failure mode. When candidate diversity is curtailed, even by a modest amount, TTS becomes much more likely to produce unsafe outputs. We present a reference-guided diversity reduction protocol (RefDiv) that serves as a diagnostic attack to stress test TTS pipelines. Through extensive experiments across open-source models (e.g. Qwen3, Mistral, Llama3.1, Gemma3) and two widely used TTS strategies (Monte Carlo Tree Search and Best-of-N), constraining diversity consistently signifies the rate at which TTS produces unsafe results. The effect is often stronger than that produced by prompts directly with high adversarial intent scores. This observed phenomenon also transfers across TTS strategies and to closed-source models (e.g. OpenAI o3-mini and Gemini-2.5-Pro), thus indicating that this is a general and extant property of TTS rather than a model-specific artifact. Additionally, we find that numerous widely used safety guardrail classifiers (e.g. Llama-Guard), are unable to flag the adversarial input prompts generated by RefDiv, demonstrating that existing defenses offer limited protection against this diversity-driven failure mode.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 44 of its 53 references that are in the map. First posted on arXiv 4 Oct 2025.

Builds on

44 of 53 references.

Closest ancestors · 7

Same direction, newest first

  1. 2025Enhancing Test-Time Scaling of Large Language Models with Hierarchical Retrieval-Augmented MCTS8 Jul
  2. 2025Wider or Deeper? Scaling LLM Inference-Time Compute with Adaptive Branching Tree Search6 Mar
  3. 2025s1: Simple test-time scaling31 Jan
  4. 2024Forest-of-Thought: Scaling Test-Time Compute for Enhancing LLM Reasoning12 Dec
  5. 2024Interpretable Contrastive Monte Carlo Tree Search Reasoning2 Oct
  6. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Models17 May
  7. 2023Self-Refine: Iterative Refinement with Self-Feedback30 Mar

Other references · 37

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  3. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  4. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  5. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  6. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  7. 2023Let's Verify Step by Stepcited by 3,598
  8. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  9. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  10. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
  11. 2021DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharingcited by 1,219
  12. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,124
25 more
  1. 2022Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questionscited by 911
  2. 2019DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generationcited by 842
  3. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Modelscited by 738
  4. 2023A Comprehensive Overview of Large Language Modelscited by 639
  5. 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automaticallycited by 619
  6. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 608
  7. 2025Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAscited by 491
  8. 2023Large Language Models Are State-of-the-Art Evaluators of Translation Qualitycited by 416
  9. 2023DeepInception: Hypnotize Large Language Model to Be Jailbreakercited by 337
  10. 2023A Wolf in Sheep's Clothing: Generalized Nested Jailbreak Prompts can Fool Large Language Models Easilycited by 242
  11. 2024Monte Carlo Tree Search Boosts Reasoning via Iterative Preference Learningcited by 238
  12. 2024ArtPrompt: ASCII Art-based Jailbreak Attacks against Aligned LLMscited by 197
  13. 2024AutoDAN-Turbo: A Lifelong Agent for Strategy Self-Exploration to Jailbreak LLMscited by 185
  14. 2024MapCoder: Multi-Agent Code Generation for Competitive Problem Solvingcited by 177
  15. 2025OverThink: Slowdown Attacks on Reasoning LLMscited by 94
  16. 2025H-CoT: Hijacking the Chain-of-Thought Safety Reasoning Mechanism to Jailbreak Large Reasoning Models, Including OpenAI o1/o3, DeepSeek-R1, and Gemini 2.0 Flash Thinkingcited by 94
  17. 2024FlipAttack: Jailbreak LLMs via Flippingcited by 73
  18. 2024Calibrating Language Models with Adaptive Temperature Scalingcited by 57
  19. 2024Pack of LLMs: Model Fusion at Test-Time via Perplexity Optimizationcited by 56
  20. 2025A Mousetrap: Fooling Large Reasoning Models for Jailbreak with Chain of Iterative Chaoscited by 39
  21. 2024Ensembling Large Language Models with Process Reward-Guided Tree Search for Better Complex Reasoningcited by 38
  22. 2025MCTS-Judge: Test-Time Scaling in LLM-as-a-Judge for Code Correctness Evaluationcited by 32
  23. 2020Plug-and-Play Conversational Modelscited by 30
  24. 2024Preemptive Answer "Attacks" on Chain-of-Thought Reasoningcited by 16
  25. 2025Watching the AI Watchdogs: A Fairness and Robustness Analysis of AI Safety Moderation Classifierscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2025Agentic AI Security: Threats, Defenses, Evaluation, and Open Challengescited by 26
  2. 2026Revisiting the Effectiveness of LLM Pruning for Test-Time Scalingcited by 2

Similar, not cited

Close in wording, no citation either way

  1. 2025Revisiting Test-Time Scaling: A Survey and a Diversity-Aware Method for Efficient Reasoning
  2. 2025Mitigating Strategy-Selection Bias in Reasoning for More Effective Test-Time Scaling
  3. 2025The Art of Scaling Test-Time Compute for Large Language Models
  4. 2025Are We Scaling the Right Thing? A System Perspective on Test-Time Scaling
  5. 2026Exposing Long-Tail Safety Failures in Large Language Models through Efficient Diverse Response Sampling
  6. 2025On the Effect of Sampling Diversity in Scaling LLM Inference
  7. 2026Test-Time Scaling in the Wild: Why Exploitation, Not Exploration, Is the Bottleneck
  8. 2025A Survey on Test-Time Scaling in Large Language Models: What, How, Where, and How Well?