Category Archives: Okategoriserade

Why it is a bad idea to have test coverage targets

 

A lot has previously been written on this topic, but it seems like there is a need for another post in this area. Probably saying the same thing as the other ones.

Test coverage is a negative metric. While a low test coverage is bad it is not certain that a high test coverage is  a good thing. It really depends on the tests. What we are testing and how well the tests are implemented. A 100% test coverage with tests that are all green is not so usable if the tests for instance doesn’t have any assertions. Test coverage tells you if there are parts of the system that are lacking tests. Not how good your tests suite is. A high coverage score might also create a sense of false security, that will prevent a team from doing what’s needed to create a product with good quality.

kermit

For a team under pressure it might be tempting to start gaming the system to reach a test coverage target that has been forced upon them. Testing things that are easy to test. Instead of writing tests that are useful, but hard to implement.

So when do we have enough test coverage. According to Martin Fowler it is when we have reached the following state.

  • We have a low fault slip through to production.
  • You are not afraid of changing the code.

Sadly the feeling in your stomach while delivering is not as easy to share with upper management as your test coverage result would be.

Read some of Martin Fowler’s thoughts on this topic here.

 

Advertisements