# MuMuTestUp: Mutation-based Multi-Agent Test Case Update

arXiv: https://arxiv.org/abs/2605.19265 · HTML: https://mlsanity.com/paper/2605.19265/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use.
- Builds on 12 of its 44 references that are in the map.
- First posted on arXiv 19 May 2026.

## Placed in

- LLM agents and tool use / [Software engineering coding agents](https://mlsanity.com/agents/software-engineering-coding-agents/)

## Builds on (most cited first, 12 of 12)

- [SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering](https://mlsanity.com/paper/2405.15793/) (2024)
- [Same Task, More Tokens: the Impact of Input Length on the Reasoning Performance of Large Language Models](https://mlsanity.com/paper/2402.14848/) (2024)
- [Effective Test Generation Using Pre-trained Large Language Models and Mutation Testing](https://mlsanity.com/paper/2308.16557/) (2023)
- [Pynguin: Automated Unit Test Generation for Python](https://mlsanity.com/paper/2202.05218/) (2022)
- [HITS: High-coverage LLM-based Unit Test Generation via Method Slicing](https://mlsanity.com/paper/2408.11324/) (2024)
- [Mutation-Guided LLM-based Test Generation at Meta](https://mlsanity.com/paper/2501.12862/) (2025)
- [Exploring the Capabilities of LLMs for Code Change Related Tasks](https://mlsanity.com/paper/2407.02824/) (2024)
- [Effects of Prompt Length on Domain-specific Tasks for Large Language Models](https://mlsanity.com/paper/2502.14255/) (2025)
- [An Empirical Study on Prompt Compression for Large Language Models](https://mlsanity.com/paper/2505.00019/) (2025)
- [REACCEPT: Automated Co-evolution of Production and Test Code Based on Dynamic Validation and Large Language Models](https://mlsanity.com/paper/2411.11033/) (2024)
- [Fix the Tests: Augmenting LLMs to Repair Test Cases with Static Collector and Neural Reranker](https://mlsanity.com/paper/2407.03625/) (2024)
- [Unit Test Update through LLM-Driven Context Collection and Error-Type-Aware Refinement](https://mlsanity.com/paper/2509.24419/) (2025)

## Abstract (arXiv)

Modern software systems evolve rapidly under CI/CD practices, where tests are critical for quality. However, substantial code changes often render existing test cases obsolete, causing pipeline disruptions, reduced productivity, and compromised quality. Recent automatic test update approaches leverage LLMs to refine test cases via execution feedback and exact-matching context retrieval, prioritizing executability and line coverage but suffering three limitations: (1) neglecting test assertion adequacy, weakening fault detection; (2) relying on coarse line coverage instead of specific uncovered lines/branches; (3) using exact-matching retrieval, which fails for LLM hallucinated queries. To address these, we propose MuMuTestUp, a mutation-guided multi-agent framework with three specialized agents: Mutation Analysis (strengthens assertions via surviving mutants), Coverage Analysis (generates targeted repair instructions for uncovered lines/branches), and Semantic Retrieval (handles hallucinations via semantic-similarity search). We also construct PRBENCH, a 571-sample pull-request-level dataset from 10 open-source Java projects (validated for cross-commit update scenarios). Evaluations against state-of-the-art baselines use both open-source (Deepseek-V3.2) and closed-source (GPT-4.1) LLMs.