Over the next few weeks, I will blog on the cost benefits of unit testing and how to quantify such benefits. Commonly quoted benefits of unit testing are , , , :
- Problems are found early in the development cycle.
- Code that works now, will work in the future.
- New features will not break existing functionality.
- Making change becomes easier, as controls are in place.
- The development process becomes more flexible.
- Implementation design is improved as APIs are forced to be more flexible and unit-testable.
- Bringing new developers on board becomes easier and improve teamwork. Unit tests document the code.
- The need for manual testing is reduced.
- The development process becomes more predictable and repeatable
bug indigestion. Unit testing is critical to prevent unit-level issues to be uncovered later in the software development life cycle. Software methodologies that rely heavily on developer unit-level testing can therefore achieve a much lower cost of ownership throughout the software development life cycle.
Enforcing unit-level testing throughout the development process should be a major focus of any management team involved in managing the delivery of a software product or solution. Identifying the optimal amount of unit-level testing to maximize the benefits of writing unit tests is hard to measure. Best practices suggest that the ratio of test code to code under test required to achieve at least 90% code coverage is between 2/1 and 4/1. This means that to thoroughly test a 100-line Java class requires 200 to 400 lines of test code. The higher the unit-level test coverage, the better the quality. This suggests a best case scenario and does not necessarily maximizes the return on writing unit tests. My experience suggests that the benefits of unit testing can be achieved much sooner but also suggests that there seems to be a threshold above which the benefits of unit testing can be most felt. Let's take, as an example one of the projects I recently completed. As illustrated below, the initial phases of QA resulted in a flood of bugs. At the same time, levels of unit tests were insufficient. I would draw a first lesson from this observation:
To maximize, the cost benefits of unit testing; start writing tests early. Focus on "quality" assertions where assertions are performed against relevant data.
As testing went on, I believe that we can clearly identify the point of inflexion where the value of unit tests really shows. As illustrated below, as our unit tests level improved, our development team was able to significantly improve our bugs closing rate without introducing new issues and therefore avoid a bugterial infection. Therefore, the second key lessons:
Good unit-level test coverage allows projects to significantly shorten their bug-fixing cycle. Therefore, resulting in direct cost benefits.
Quantifying cost benefits depends on the type of project under scrutiny, however simple empirical data clearly demonstrates such benefits. As a result if you are a developer, you should be writing unit tests today; if you are a manager, you should be driving adoption of unit-level testing practices. The quality of your software and your ability to respond to your customers demand will significantly improve. Be agile, today!