mlsanityData to 7 Oct 2026

Automated Unit Test Improvement using Large Language Models at Meta

arXiv 2402.09171 ↗First posted 14 Feb 2024cs.SEcited by 73

Not on the map. Why?

Cited by 28 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

This paper describes Meta's TestGen-LLM tool, which uses LLMs to automatically improve existing human-written tests. TestGen-LLM verifies that its generated test classes successfully clear a set of filters that assure measurable improvement over the original test suite, thereby eliminating problems due to LLM hallucination. We describe the deployment of TestGen-LLM at Meta test-a-thons for the Instagram and Facebook platforms. In an evaluation on Reels and Stories products for Instagram, 75% of TestGen-LLM's test cases built correctly, 57% passed reliably, and 25% increased coverage. During Meta's Instagram and Facebook test-a-thons, it improved 11.5% of all classes to which it was applied, with 73% of its recommendations being accepted for production deployment by Meta software engineers. We believe this is the first report on industrial scale deployment of LLM-generated code backed by such assurances of code improvement.

Key facts. Cited by 73 papers in the map (data through 7 Oct 2026). First posted on arXiv 14 Feb 2024.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 16; 2025: 38; 2026: 19.

73 papers.

Sort:

By area:

  1. 2024RepairAgent: An Autonomous, LLM-Based Agent for Program Repaircited by 225
  2. 2024SWT-Bench: Testing and Validating Real-World Bug-Fixes with Code Agentscited by 129
  3. 2025SWE-PolyBench: A multi-language benchmark for repository level evaluation of coding agentscited by 80
  4. 2024You Name It, I Run It: An LLM Agent to Execute Tests of Arbitrary Projectscited by 45
  5. 2024CoverUp: Effective High Coverage Test Generation for Pythoncited by 36
  6. 2025Learning to Solve and Verify: A Self-Play Framework for Code and Test Generationcited by 34
  7. 2025CodeJudgeBench: Benchmarking LLM-as-a-Judge for Coding Taskscited by 32
  8. 2025AI Agentic Programming: A Survey of Techniques, Challenges, and Opportunitiescited by 26
  9. 2024The Future of AI-Driven Software Engineeringcited by 23
  10. 2025Mutation-Guided LLM-based Test Generation at Metacited by 23
  11. 2025Retrieval-Augmented Code Generation: A Survey with Focus on Repository-Level Approachescited by 20
  12. 2025debug-gym: A Text-Based Environment for Interactive Debuggingcited by 19
38 more
  1. 2025CoCoEvo: Co-Evolution of Programs and Test Cases to Enhance Code Generationcited by 14
  2. 2025Migrating Code At Scale With LLMs At Googlecited by 14
  3. 2025Metamorphic Testing of Large Language Models for Natural Language Processingcited by 12
  4. 2024Leveraging Large Language Models for Enhancing the Understandability of Generated Unit Testscited by 11
  5. 2024LLMs: A Game-Changer for Software Engineers?cited by 11
  6. 2025Learning to Generate Unit Test via Adversarial Reinforcement Learningcited by 11
  7. 2025Rethinking the Evaluation of Secure Code Generationcited by 10
  8. 2024VulnLLMEval: A Framework for Evaluating Large Language Models in Software Vulnerability Detection and Patchingcited by 9
  9. 2025LLM Test Generation via Iterative Hybrid Program Analysiscited by 9
  10. 2024RAGProbe: An Automated Approach for Evaluating RAG Applicationscited by 7
  11. 2025CLOVER: A Test Case Generation Benchmark with Coverage, Long-Context, and Verificationcited by 7
  12. 2025Evaluating LLM-Based Regression Test Generationcited by 6
  13. 2026Test vs Mutant: Adversarial LLM Agents for Robust Unit Test Generationcited by 6
  14. 2025Disproving Program Equivalence with LLMscited by 5
  15. 2025How Safe Are AI-Generated Patches? A Large-scale Study on Security Risks in LLM and Agentic Automated Program Repair on SWE-benchcited by 5
  16. 2026Are Coding Agents Generating Over-Mocked Tests? An Empirical Studycited by 5
  17. 2024Automating REST API Postman Test Cases Using LLMcited by 4
  18. 2025Unveiling the Landscape of LLM Deployment in the Wild: An Empirical Studycited by 4
  19. 2026Do Autonomous Agents Contribute Test Code? A Study of Tests in Agentic Pull Requestscited by 4
  20. 2024"You still have to study" -- On the Security of LLM generated codecited by 3
  21. 2025AI-Driven Tools in Modern Software Quality Assurance: An Assessment of Benefits, Challenges, and Future Directionscited by 3
  22. 2025An Agent-Based Framework for the Automatic Validation of Mathematical Optimization Modelscited by 3
  23. 2024Test Code Generation for Telecom Software Systems using Two-Stage Generative Modelcited by 2
  24. 2024Generative transformations and patterns in LLM-native approaches for software verification and falsificationcited by 2
  25. 2024Design choices made by LLM-based test generators prevent them from finding bugscited by 2
  26. 2025QiMeng-MuPa: Mutual-Supervised Learning for Sequential-to-Parallel Code Translationcited by 2
  27. 2025STAF: Leveraging LLMs for Automated Attack Tree-Based Security Test Generationcited by 2
  28. 2025Large Language Models for Software Testing: A Research Roadmapcited by 2
  29. 2026Understanding LLM-Driven Test Oracle Generationcited by 2
  30. 2026Evaluating LLM-Based Test Generation Under Software Evolutioncited by 2
  31. 2024AutoTest: Evolutionary Code Solution Selection with Test Casescited by 1
  32. 2025Test Amplification for REST APIs via Single and Multi-Agent LLM Systemscited by 1
  33. 2025Harden and Catch for Just-in-Time Assured LLM-Based Software Testing: Open Research Challengescited by 1
  34. 2025SOCIA-nabla: Textual Gradient Meets Multi-Agent Orchestration for Automated Simulator Generationcited by 1
  35. 2025Towards Functional Correctness of Large Code Models with Selective Generationcited by 1
  36. 2025LLMs for Automated Unit Test Generation and Assessment in Java: The AgoneTest Frameworkcited by 1
  37. 2026From LLMs to Agents in Programming: The Impact of Providing an LLM with a Compilercited by 1
  38. 2026Just-in-Time Catching Test Generation at Metacited by 1

50 most cited of 73 All 73 (JSON)