Generating Unit Tests for Documentation

Software projects capture information in various kinds of artifacts, including source code, tests, and documentation. Such artifacts routinely encode information that is redundant, i.e., when a specification encoded in the source code is also separately tested and documented. Without supporting technology, such redundancy easily leads to inconsistencies and a degradation of documentation quality. We designed a tool-supported technique, called DScribe, that leverages redundancy between tests and documentation to generate consistent and checkable documentation and unit tests based on a single source of information. DScribe generates unit tests and documentation fragments based on a novel template and artifact generation technology. By pairing tests and documentation generation, DScribe provides a mechanism to automatically detect and replace outdated documentation. Our evaluation of the Apache Commons IO library revealed that of 835 specifications about exception handling, 85% of them were not tested or correctly documented, and DScribe could be used to automatically generate 97% of the tests and documentation.