# On Learning Meaningful Assert Statements for Unit Test Cases

arXiv: https://arxiv.org/abs/2002.05800 · HTML: https://mlsanity.com/paper/2002.05800/

## Key facts

- Cited by 42 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Feb 2020.

## Built on by (most cited first, 30 of 42)

- [LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code](https://mlsanity.com/paper/2403.07974/) (2024)
- [An Empirical Evaluation of Using Large Language Models for Automated Unit Test Generation](https://mlsanity.com/paper/2302.06527/) (2023)
- [RLEF: Grounding Code LLMs in Execution Feedback with Reinforcement Learning](https://mlsanity.com/paper/2410.02089/) (2024)
- [Evaluating Instruction-Tuned Large Language Models on Code Comprehension and Generation](https://mlsanity.com/paper/2308.01240/) (2023)
- [Learning Deep Semantics for Test Completion](https://mlsanity.com/paper/2302.10166/) (2023)
- [CAT-LM: Training Language Models on Aligned Code And Tests](https://mlsanity.com/paper/2310.01602/) (2023)
- [Deep Learning Meets Software Engineering: A Survey on Pre-Trained Models of Source Code](https://mlsanity.com/paper/2205.11739/) (2022)
- [Benchmarking Causal Study to Interpret Large Language Models for Source Code](https://mlsanity.com/paper/2308.12415/) (2023)
- [Toward a Theory of Causation for Interpreting Neural Code Models](https://mlsanity.com/paper/2302.03788/) (2023)
- [Towards More Trustworthy and Interpretable LLMs for Code through Syntax-Grounded Explanations](https://mlsanity.com/paper/2407.08983/) (2024)
- [Evaluating and Explaining Large Language Models for Code Using Syntactic Structures](https://mlsanity.com/paper/2308.03873/) (2023)
- [LLM Test Generation via Iterative Hybrid Program Analysis](https://mlsanity.com/paper/2503.13580/) (2025)
- [TestForge: Feedback-Driven, Agentic Test Suite Generation](https://mlsanity.com/paper/2503.14713/) (2025)
- [Domain Adaptation for Code Model-based Unit Test Case Generation](https://mlsanity.com/paper/2308.08033/) (2023)
- [Assessing Evaluation Metrics for Neural Test Oracle Generation](https://mlsanity.com/paper/2310.07856/) (2023)
- [Perfect is the enemy of test oracle](https://mlsanity.com/paper/2302.01488/) (2023)
- [Generating executable oracles to check conformance of client code to requirements of JDK Javadocs using LLMs](https://mlsanity.com/paper/2411.01789/) (2024)
- [How Propense Are Large Language Models at Producing Code Smells? A Benchmarking Study](https://mlsanity.com/paper/2412.18989/) (2024)
- [Evaluating LLM-Based Regression Test Generation](https://mlsanity.com/paper/2501.11086/) (2025)
- [Revisiting and Improving Retrieval-Augmented Deep Assertion Generation](https://mlsanity.com/paper/2309.10264/) (2023)
- [UniTSyn: A Large-Scale Dataset Capable of Enhancing the Prowess of Large Language Models for Program Testing](https://mlsanity.com/paper/2402.03396/) (2024)
- [exLong: Generating Exceptional Behavior Tests with Large Language Models](https://mlsanity.com/paper/2405.14619/) (2024)
- [ASSERTIFY: Utilizing Large Language Models to Generate Assertions for Production Code](https://mlsanity.com/paper/2411.16927/) (2024)
- [Improving Deep Assertion Generation via Fine-Tuning Retrieval-Augmented Pre-trained Language Models](https://mlsanity.com/paper/2502.16071/) (2025)
- [Benchmarking AI Models in Software Engineering: A Review, Search Tool, and Unified Approach for Elevating Benchmark Quality](https://mlsanity.com/paper/2503.05860/) (2025)
- [FuzzAug: Data Augmentation by Coverage-guided Fuzzing for Neural Test Generation](https://mlsanity.com/paper/2406.08665/) (2024)
- [SURGE: On the Potential of Large Language Models as General-Purpose Surrogate Code Executors](https://mlsanity.com/paper/2502.11167/) (2025)
- ["I'm Not Reading All of That": Understanding Software Engineers' Level of Cognitive Engagement with Agentic Coding Assistants](https://mlsanity.com/paper/2603.14225/) (2026)
- [Evaluating the Effectiveness and Efficiency of Demonstration Retrievers in RAG for Coding Tasks](https://mlsanity.com/paper/2410.09662/) (2024)
- [Improving Retrieval-Augmented Deep Assertion Generation via Joint Training](https://mlsanity.com/paper/2502.10696/) (2025)

## Abstract (arXiv)

Software testing is an essential part of the software lifecycle and requires a substantial amount of time and effort. It has been estimated that software developers spend close to 50% of their time on testing the code they write. For these reasons, a long standing goal within the research community is to (partially) automate software testing. While several techniques and tools have been proposed to automatically generate test methods, recent work has criticized the quality and usefulness of the assert statements they generate. Therefore, we employ a Neural Machine Translation (NMT) based approach called Atlas(AuTomatic Learning of Assert Statements) to automatically generate meaningful assert statements for test methods. Given a test method and a focal method (i.e.,the main method under test), Atlas can predict a meaningful assert statement to assess the correctness of the focal method. We applied Atlas to thousands of test methods from GitHub projects and it was able to predict the exact assert statement manually written by developers in 31% of the cases when only considering the top-1 predicted assert. When considering the top-5 predicted assert statements, Atlas is able to predict exact matches in 50% of the cases. These promising results hint to the potential usefulness ofour approach as (i) a complement to automatic test case generation techniques, and (ii) a code completion support for developers, whocan benefit from the recommended assert statements while writing test code.