# PseudoBench: Measuring How Agentic Auto-Research Fuels Pseudoscience

arXiv: https://arxiv.org/abs/2606.18060 · HTML: https://mlsanity.com/paper/2606.18060/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use.
- Builds on 19 of its 59 references that are in the map.
- First posted on arXiv 16 Jun 2026.

## Placed in

- LLM agents and tool use / [Web, search, and research agents](https://mlsanity.com/agents/web-search-and-research-agents/)

## Builds on (most cited first, 19 of 19)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Constitutional AI: Harmlessness from AI Feedback](https://mlsanity.com/paper/2212.08073/) (2022)
- [Towards Understanding Sycophancy in Language Models](https://mlsanity.com/paper/2310.13548/) (2023)
- [Discovering Language Model Behaviors with Model-Written Evaluations](https://mlsanity.com/paper/2212.09251/) (2022)
- [How Language Model Hallucinations Can Snowball](https://mlsanity.com/paper/2305.13534/) (2023)
- [Towards End-to-End Automation of AI Research](https://mlsanity.com/paper/2606.15497/) (2026)
- [ELEPHANT: Measuring and understanding social sycophancy in LLMs](https://mlsanity.com/paper/2505.13995/) (2025)
- [Sycophancy in Large Language Models: Causes and Mitigations](https://mlsanity.com/paper/2411.15287/) (2024)
- [The Factual Inconsistency Problem in Abstractive Text Summarization: A Survey](https://mlsanity.com/paper/2104.14839/) (2021)
- [Are You Sure? Challenging LLMs Leads to Performance Drops in The FlipFlop Experiment](https://mlsanity.com/paper/2311.08596/) (2023)
- [When Large Language Models contradict humans? Large Language Models' Sycophantic Behaviour](https://mlsanity.com/paper/2311.09410/) (2023)
- [EvoScientist: Towards Multi-Agent Evolving AI Scientists for End-to-End Scientific Discovery](https://mlsanity.com/paper/2603.08127/) (2026)
- [El Agente: An Autonomous Agent for Quantum Chemistry](https://mlsanity.com/paper/2505.02484/) (2025)
- [Position: The AI Conference Peer Review Crisis Demands Author Feedback and Reviewer Rewards](https://mlsanity.com/paper/2505.04966/) (2025)
- [Exploring Autonomous Agents through the Lens of Large Language Models: A Review](https://mlsanity.com/paper/2404.04442/) (2024)
- [LabSafety Bench: Benchmarking LLMs on Safety Issues in Scientific Labs](https://mlsanity.com/paper/2410.14182/) (2024)
- [Sycophantic AI makes human interaction feel more effortful and less satisfying over time](https://mlsanity.com/paper/2605.07912/) (2026)
- [How Far Are We From True Auto-Research?](https://mlsanity.com/paper/2605.19156/) (2026)
- [UAQFact: Evaluating Factual Knowledge Utilization of LLMs on Unanswerable Questions](https://mlsanity.com/paper/2505.23461/) (2025)

## Abstract (arXiv)

As Large Language Model based agents enter autonomous scientific research, their ability to resist pseudoscience becomes increasingly important. Otherwise, such systems may rapidly generate plausible yet misleading studies that contaminate academic literature and erode trust in science. We present PseudoBench, an adversarial benchmark for evaluating whether agentic auto-research systems can identify and resist pseudoscientific narratives. PseudoBench contains 200 curated pseudoscientific claim-evidence pairs across five domains and evaluates agents through an end-to-end research pipeline from experiments to writing. Testing seven state-of-the-art agents, we find that current systems readily produce persuasive reports that align with pseudoscientific premises with near-zero refusal rates and the highest resistance of only 27.4%. Stronger agents risk packaging pseudoscience in more sophisticated scientific language, increasing its apparent credibility. These findings reveal an alarming capacity to fuel pseudoscience, calling for scientific alignment before widespread deployment.