# Test vs Mutant: Adversarial LLM Agents for Robust Unit Test Generation

arXiv: https://arxiv.org/abs/2602.08146 · HTML: https://mlsanity.com/paper/2602.08146/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use.
- Builds on 31 of its 73 references that are in the map.
- First posted on arXiv 8 Feb 2026.

## Placed in

- LLM agents and tool use / [Software engineering coding agents](https://mlsanity.com/agents/software-engineering-coding-agents/)

## Builds on (most cited first, 30 of 31)

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [An Empirical Evaluation of Using Large Language Models for Automated Unit Test Generation](https://mlsanity.com/paper/2302.06527/) (2023)
- [SWE-rebench: An Automated Pipeline for Task Collection and Decontaminated Evaluation of Software Engineering Agents](https://mlsanity.com/paper/2505.20411/) (2025)
- [Unit Test Case Generation with Transformers and Focal Context](https://mlsanity.com/paper/2009.05617/) (2020)
- [ChatUniTest: A Framework for LLM-Based Test Generation](https://mlsanity.com/paper/2305.04764/) (2023)
- [Automated Unit Test Improvement using Large Language Models at Meta](https://mlsanity.com/paper/2402.09171/) (2024)
- [SWE-Exp: Experience-Driven Software Issue Resolution](https://mlsanity.com/paper/2507.23361/) (2025)
- [Using Large Language Models to Generate JUnit Tests: An Empirical Study](https://mlsanity.com/paper/2305.00418/) (2023)
- [SWE-Debate: Competitive Multi-Agent Debate for Software Issue Resolution](https://mlsanity.com/paper/2507.23348/) (2025)
- [Effective Test Generation Using Pre-trained Large Language Models and Mutation Testing](https://mlsanity.com/paper/2308.16557/) (2023)
- [Pynguin: Automated Unit Test Generation for Python](https://mlsanity.com/paper/2202.05218/) (2022)
- [From Code to Correctness: Closing the Last Mile of Code Generation with Hierarchical Debugging](https://mlsanity.com/paper/2410.01215/) (2024)
- [SWE-Pruner: Self-Adaptive Context Pruning for Coding Agents](https://mlsanity.com/paper/2601.16746/) (2026)
- [LongCodeZip: Compress Long Context for Code Language Models](https://mlsanity.com/paper/2510.00446/) (2025)
- [CoverUp: Effective High Coverage Test Generation for Python](https://mlsanity.com/paper/2403.16218/) (2024)
- [HITS: High-coverage LLM-based Unit Test Generation via Method Slicing](https://mlsanity.com/paper/2408.11324/) (2024)
- [WhiteFox: White-Box Compiler Fuzzing Empowered by Large Language Models](https://mlsanity.com/paper/2310.15991/) (2023)
- [A3Test: Assertion-Augmented Automated Test Case Generation](https://mlsanity.com/paper/2302.10352/) (2023)
- [Learning Deep Semantics for Test Completion](https://mlsanity.com/paper/2302.10166/) (2023)
- [CAT-LM: Training Language Models on Aligned Code And Tests](https://mlsanity.com/paper/2310.01602/) (2023)
- [Between Lines of Code: Unraveling the Distinct Patterns of Machine and Human Programmers](https://mlsanity.com/paper/2401.06461/) (2024)
- [Mutation-Guided LLM-based Test Generation at Meta](https://mlsanity.com/paper/2501.12862/) (2025)
- [In Line with Context: Repository-Level Code Generation via Context Inlining](https://mlsanity.com/paper/2601.00376/) (2026)
- [GlimpRouter: Efficient Collaborative Inference by Glimpsing One Token of Thoughts](https://mlsanity.com/paper/2601.05110/) (2026)
- [Leveraging Large Language Models for Enhancing the Understandability of Generated Unit Tests](https://mlsanity.com/paper/2408.11710/) (2024)
- [LLMs as Continuous Learners: Improving the Reproduction of Defective Code in Software Issues](https://mlsanity.com/paper/2411.13941/) (2024)
- [Extracting Concise Bug-Fixing Patches from Human-Written Patches in Version Control Systems](https://mlsanity.com/paper/2103.00156/) (2021)
- [TestForge: Feedback-Driven, Agentic Test Suite Generation](https://mlsanity.com/paper/2503.14713/) (2025)
- [A Comprehensive Study on Large Language Models for Mutation Testing](https://mlsanity.com/paper/2406.09843/) (2024)

## Built on by (most cited first, 7 of 7)

- [EffiSkill: Agent Skill Based Automated Code Efficiency Optimization](https://mlsanity.com/paper/2603.27850/) (2026)
- [LLM Agents Can See Code Repositories](https://mlsanity.com/paper/2606.14061/) (2026)
- [Repo0: Design-Driven Zero-to-All Code Generation](https://mlsanity.com/paper/2608.19854/) (2026)
- [SkillForge: Self-Distilling Agents for Project-Specific Issue Resolution](https://mlsanity.com/paper/2608.18933/) (2026)
- [Schr"odinger's Code Repository: Have LLMs Learned SWE-bench or Memorized It?](https://mlsanity.com/paper/2609.27891/) (2026)
- [Beyond Rule-Based Mutation Testing: Test-Aware Mutant Generation Using Large Language Models](https://mlsanity.com/paper/2609.35841/) (2026)
- [GTDD: Generative Test-Driven Development for AI Coding Agents with Adversarial Testing](https://mlsanity.com/paper/2610.02952/) (2026)

## Abstract (arXiv)

Software testing is a critical, yet resource-intensive phase of the software development lifecycle. Over the years, various automated tools have been developed to aid in this process. Search-based approaches typically achieve high coverage but produce tests with low readability, whereas large language model (LLM)-based methods generate more human-readable tests but often suffer from low coverage and compilability. While the majority of research efforts have focused on improving test coverage and readability, little attention has been paid to enhancing the robustness of bug detection, particularly in exposing corner cases and vulnerable execution paths. To address this gap, we propose AdverTest, a novel adversarial framework for LLM-powered test case generation. AdverTest comprises two interacting agents: a test case generation agent (T) and a mutant generation agent (M). These agents engage in an adversarial loop, where M persistently creates new mutants "hacking" the blind spots of T's current test suite, while T iteratively refines its test cases to "kill" the challenging mutants produced by M. This interaction loop is guided by both coverage and mutation scores, enabling the system to co-evolve toward both high test coverage and bug detection capability. Experimental results in the Defects4J dataset show that our approach improves fault detection rates by 8.56% over the best existing LLM-based methods and by 63.30% over EvoSuite, while also improving line and branch coverage.