Key Reports for Monitoring Application Development - Need for Historical Data

When managing distributed software development teams with wide ranging skills sets, code base intelligence becomes critical to ensure the quality of the ongoing development effort. Here are some lessons learned worthwhile sharing:

• Unit test code coverage should be put in historical context: Tools like Cobertura and Clover provide great unit test coverage reports; however, most default reports provide point in time coverage and as a result become difficult to use as a metrics for the development effort. To create a successful developer testing practice, developers activities should be measured against specific targets. Setting developers unit testing targets is a great practice to foster the creation and development of unit tests as an implicit and routine part of their activity. Historical measurement is critical to be able to manage such activity efficiently. Clover provides such historical coverage report.
• Unit tests are necessary but beware of bad tests: Enforcing unit test coverage is important to facilitate future development work and improve code quality (see previous post), however poorly written unit tests can provide a fall sense of confidence that proper checks are in place. Unit tests that provide poor assertions checks will result in reasonable code coverage but will not provide the proper checks for guaranteeing the code base quality. In addition to unit test reports, and code coverage reports, unit test code should comply to a specific set of rules as illustrated by the PMD Junit rules.
• Measure your developers activities: Activity reports measuring development activity can provide great insight on the evolution of a code base and the activity level of various contributors. StatCVS provides a very detailed set of statistic that can be useful to understand and monitor the activities of a large and distributed team.
• Dashboard and code quality indexes: Dashboard are critical for management to be able to measure and assess the evolution of various components of a large project. Maven provides a flexible plugin for a point in time dashboard. However, in order to properly follow the evolution of a large development project, historical data is important as it provides the ability to identify key areas of improvement as well as measurable targets. Continuous improvement and continuous refactoring often advocated by agile development methodology advocates requires good metrics to measure improvement and justify the benefits.

Those reports provide a sample set of tools that can be useful when managing development teams. However, one key issue for management is to be able to correlate metrics improvement with critical business metrics such as development effort cost savings (shortened features development time lines, decrease bug level, shortened quality control, etc). More thoughts to come on that subject...