Documentation needs to be comprehensive, always up-to-date and accessible. By comprehensive I mean that it must cover all important areas of the code as well as all functions of the application. While importance of documentation is obvious to most, many struggle without success to have it accurate and up-to-date. Response to “poor” documentation is often assignment of more resources and more time. More often than not, documentation is created for wrong reasons.
Reasons for requesting documentation
Documentation can be requested for various reasons. Teams are often asked to work on documentation for political reasons or because of sheer ignorance. Some of the wrong reasons to create documentation can be:
- Someone thinks that some document is related to the project success
- Documentation justifies someone’s existence
- Requester does not know any better
- Requester wants reassurance that everything is OK
- Process says that document should be created
Documentation is not up-to-date
Main problem with software documentation is that it is not up-to-date most of the time. As soon as some part of the code changes, documentation stops reflecting the actual situation. This statement applies to almost any type of documentation with requirements and test cases being the most affected. No matter how hard we try, documentation inevitably gets outdated.
Who uses the documentation?
Depending on the audience, type of needed documentation and its format varies considerably. Developers, testers, customers, managers and end users are probably major profiles of potential documentation consumers.
Developers
Developers shouldn’t rely on system documentation because it is almost never up-to-date. Besides, no documentation can provide as detailed and up-to-date description of the code as the code itself. If you want to see what some method does, take a look at the method. Not sure what some class does? Take a look at the class. Necessity to document code is often a sign that the code itself is not well written.
Using code as documentation does not exclude other types of documents. The key is to avoid duplication. If details of the system can be obtained by reading the code, other types of documentation can provide quick guidelines and high level overview. Non-code documentation should answer questions like what the general purpose of the system is and what technologies are used by the system. In many cases, simple README.md is enough to provide a quick start that developers need. Sections like project description, environment setup, installation, build and packaging instructions are very helpful for newcomers. From there on, code is the bible. Production code provides all needed details while test code acts as the description of the intent behind the production code. Tests are executable documentation with TDD being the most common way to create and maintain it.
Assuming that some form of continuous integration is in use, if some part of test-documentation is incorrect, it will fail and be fixed soon afterwards. CI fixes the problem of incorrect test-documentation but it does not ensure that all functionality is documented. For that reason (among many others) test-documentation should be created in the TDD fashion. If all functionality is defined as tests before the implementation code and execution of all tests is successful, than tests act as complete and up-to-date documentation that can be used by developers.
What should we do with the rest of the team? Testers, customers, managers and other non-coders might not be able to obtain needed information from the production and test code.
Testers
Two most common types of testing are black and white box testing. This division is important since it also divides testers into those who do know how to write or at least read code (white-box testing) and those who don’t (black-box testing). In some cases testers can do both types. However, more often than not, testers do not know how to code so the documentation that is usable for developers is not usable for them. If documentation needs to be decoupled from the code, unit tests are not a good match. That is one of the reasons why BDD came to being. It can provide documentation necessary for non-coders while still maintaining advantages of TDD and automation.
Customers
Customers need to be able to define new functionality of the system as well as to be able to get information about all the important aspects of the current system. That documentation should not be too technical (code is not an option) but still be always up to date. BDD narratives and scenarios are one of the best ways to provide this type of documentation. Ability to act as acceptance criteria (written before the code), be executed frequently (preferably on every commit) and be written in natural language makes BDD stories not only always up-to-date but usable by those who do not want to inspect the code.
Executable documentation
Documentation is integral part of the software. As with any other part of the software, it needs to be tested often so that we’re sure that it is accurate and up-to-date. The only cost effective way to accomplish this is to have executable documentation that can be integrated into your continuous integration system. TDD as methodology is the best way to move towards this direction. On a low level, unit tests are a best fit. On the other hand, BDD provides a good way to work on a functional level while maintaining understanding accomplished using natural language.
| Reference: | Tests as documentation from our JCG partner Viktor Farcic at the Technology conversations blog. |
