The discussion about the number of code coverage you must have, seems to gain more and more attention lately. On the one side there are the people who write their tests after the fact and their managers, claiming numbers that vary more or less in the range of 60-80%. On the other side there are the people who do TDD, who wonder how you can get anything less than about 100%.

Sure thing I am from the TDD camp. But I’m not going to write another post that claims that you must have 100% code coverage. Well, I might just a bit, but I’ll provide good reasoning. Moreover I want to make the point that putting down any number of coverage that is sufficient, does not improve the actual coverage.

A project I recently worked on, the goal was set, of course after a lot of discussion, to 80%. Project managers interpreted anything less as lack of quality, and anything more as waste (or as they called it, ‘overkill’). As a result I was asked why my code was about 95% covered by tests. In their eyes I was ‘wasting time by writing more tests than necessary’. I am anything but unwilling to cut waste, and I asked what 20% of the features didn’t matter, so I could cut even more waste by skipping the tests and the production code.

As you might expect, the project manager didn’t like any bit of that question. ‘Of course the features matter, it’s just that testing them is overkill’, he said.

So why do I think discussing a target number of code coverage (goal) for a project, or worse, in general, is nonsense? First of all, the goal is not something to discuss about. The only possible goal is 100%! Sure, 60% is a milestone, and so is 80%, as is 90%. But the only goal you can ever set is 100%.

And how could it be anything else? Setting the goal to anything lower is an obvious statement that a part of your code doesn’t matter. A vicious statement that for a part of the customer’s or end users’ problems, it doesn’t matter if they get solved. If that’d be true, why waste time on solving these problems in the first place? Everything you build adds value to the system and therefore the goal is to test it, or you shouldn’t be building it in the first place.

You probably won’t ever reach 100% code coverage, though. On a class level you will, but not for an entire project. In fact, I find projects with 100% overall coverage quite suspicious. Such a project smells like the focus was on gaining test coverage rather than using tests to drive the system.

I use TDD for a while now. And I use code coverage to look for problems in my tests. Yes, that’s right, I don’t care about the number. Normally my coverage is about 95-99%, but again, I don’t care. What I care about is the feasible coverage, and make sure that I covered everything feasible. And I use code coverage tools to look for any weak spots. But that I’ll discuss in another post.

With feasible coverage I refer to the code that you can possibly test. There will be some lines of code in any project that you cannot get under test. An example is I/O in java. Any interaction with I/O requires you to handle IOException. You handle it and throw a properly named runtime exception instead of the checked one (and of course include its cause). You cannot test any of the possible I/O failures, and you should not. Oh sure, there are some ways to throw these exceptions; you can use one of the nifty mocking frameworks to apply all the reflection necessary to be able to throw that exception. But again, you should not.

It doesn’t matter that you cannot get every line of code under test. What matters is that you maintain a constant drive to improve the tests and the production code. Therefore the only sensible goal of code coverage is 100%, and again, we’ll never reach that goal.

And this is why I find the discussion about a level of coverage to be wrong. The behavior you stimulate by setting a minimum amount of code coverage, is to reach that level no matter what. This leads to writing bad tests - e.g. tests without asserts. These tests will only get in the way when you make changes to the system rather than helping you to make changes. In an earlier post I wrote about the art of testing and how the primary purpose of tests is to help you refactor. As soon as tests stop helping you modify the system, you should rethink your tests!