Wednesday, September 5, 2007

Improving Test Coverage using Code Coverage

For the past few years, the industry is undergoing a lot of advancements. Particularly, new software development model have been followed. The industry moved from waterfall model towards Agile and Iternation (incremental) model. Also, a lot of progress have been made in improving productivity by employing tools to aid development. One of the most important activity is Unit testing. Unit testing is essential to ensure that the developed code works as expected and without the unit testing the development is not complete. Unit testing has to be carried on the entire code not leaving a single line of code. It is during this phase, the developers make sure that the code will work properly as expected. There are few questions that we need to ask ourselves during unit testing
  1. Are we doing it like a black box testing without looking at the source code?
  2. Do we look at the source code during unit testing?
  3. How do we make sure that the unit testing cover maximum source code?
Let us straight away get into the answers to the above questions. To answer the first question, the unit testing is not a black box testing. Doing it like a black box testing is not effective way of testing the software at unit level. For example, in the actual source code there could be lots of conditional and branching statements and loop statements. The black box testing does not test these language specific constructs properly but it focuses only on functionality of the software.

The answer to the second question is that every developer has to look at the source code while writing a unit test plan. This is very crucial and the test cases has to be based on the source code. The developer of the source code is the best person to know about the source code and hence it is advisable and desirable that the author of the software carry out the unit testing.

Before answering the third question, let us discuss on testing effectiveness. The effectiveness of testing is the covering the entire source code with minimal test cases. When the effectiveness is high, the quality and productivity of the software will also be high. As human are prove to make mistakes, it is often essential to use tools to improve effectiveness. Code coverage is one such technique to do unit testing effectively. Code coverage pin points the area of the source code that is not tested. Most of the code coverage tools work from package level to source line number level. Once the unit testing session is over, the developer can immediately see the results and improve the test plan to make it more effective.

For example, during iteration #1, the developer will execute the test cases and run code coverage in parallel. During the unit testing or after the unit testing, the developer can take a look at the code coverage reports. Since the code coverage tool reports the coverage at line level, it is easier for the developer to add test cases then and there. Most of the tools have features to generate a comprehensive reports at desired level (application, package, class, method and line level).

There are few code coverage tools (open source and commercial) available for Java. Emma and Cobertura are the most popular code coverage tools in open source arena. Both of them are much matured and used by many developers across globe. There are lot of tutorials available for both the tools. Kindly refer respective websites for more information.

No comments: