Consistency Meets Verification: Enhancing Test Generation Quality in Large Language Models Without Ground-Truth Solutions
Not on the map. Why?
Abstract (arXiv)
Large Language Models (LLMs) have significantly advanced automated test generation, yet existing methods often rely on ground-truth code for verification, risking bug propagation and limiting applicability in test-driven development. We present ConVerTest, a novel two-stage pipeline for synthesizing reliable tests without requiring prior code implementations. ConVerTest integrates three core strategies: (i) Self-Consistency(SC) to generate convergent test cases via majority voting; (ii) Chain-of-Verification (CoVe) for iterative, reasoning-guided code refinement; and (iii) a Dual Execution Agreement to crossvalidate code and tests through consensus. Experiments on BIGCODEBENCH and LESS BASIC PYTHON PROBLEMS (LBPP) benchmarks demonstrate that ConVerTest improves test validity, line coverage, and mutation scores by up to 39%, 28%, and 18% respectively over baselines. Our findings highlight ConVerTest as a robust solution for mitigating hallucinations and enhancing the reliability of autonomous software testing agents.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). First posted on arXiv 11 Feb 2026.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2024Large Language Models as Test Case Generators: Performance Evaluation and Enhancement
- 2024SWT-Bench: Testing and Validating Real-World Bug-Fixes with Code Agents
- 2026Agent-Based Test Assertion Generation via Diverse Perspective Aggregation
- 2026TDD-Agent: Test-Driven Reasoning for Code Generation
- 2026CodeContests-O: Powering LLMs via Feedback-Driven Iterative Test Case Generation
- 2025TestForge: Feedback-Driven, Agentic Test Suite Generation
- 2026Bidirectional Empowerment of Metamorphic Testing and Large Language Models: A Systematic Survey
- 2026SWE-Proof: Can Language Models Resolve Real-World Issues with Machine-Checked Proofs?