Is 100% Code Coverage a Worthy Goal?
If 75% code coverage is good, then 100% coverage must be better right? Like so many things in the software development, it depends. It depends on what your application is doing and what are the greatest risks of failure. In the Enterprise space, the question “Is 100% code coverage a worthy goal?” should be asked far more often than it is.
Edge Cases
First off let’s dispense with some edge cases. If you’re writing code that has life or death implications then it’s of course imperative to test everything as thoroughly as possible.
Likewise if the cost of failure is exorbitant such as writing code for a Mars Rover, then by all means we should test everything. You can surely think of many other instances along these lines. This post is not targeted at these scenarios.
Identify What Matters Most
Rather we’re concerned here with more standard enterprise applications and how we should approach test coverage for them. Time pressure is a fact of life in our world. We need to prioritize our efforts towards things that matter most.
Testing is definitely important! We just need to focus on testing the most important things first. To accomplish this we need to identify our application’s most likely points of failure.
If you don’t ask yourself why are you testing code blocks periodically, you’re probably wasting time and money. If you’re anything like me, you’re convinced of the value of writing tests. Sometimes it’s easy to get carried away though.
In our quest towards an idyllic world of pure Test-Driven Development(TDD) we can lose sight of more important concerns. Based on my experience most projects in the Enterprise world are not good candidates for 100% code coverage.
We’re much better off stepping back and assessing where our code base is most vulnerable. Then we can focus our testing efforts where they really matter.
Targeted Testing
Does your app perform complex calculations? Do you rely on a number of external API calls and collate results? Will it heavily depend on UI interactions or need to process uncontrolled file inputs? Asking these sorts of questions can help us focus our testing on the most likely source of system bugs and errors.
For instance if we’re working on bank software where extreme precision is required in calculating financial transactions, we’re definitely going to want unit tests to validate. Do we rely on a number of external API calls? Then our integration test coverage will need to be significant.
Building a robust test suite covering the main risks of our given application is the real goal here. That’s where the value of testing will shine through. Not by hitting an arbitrary test coverage metric that’s good for little else than as a mottled feather in a manager’s cap.
Testing Antipatterns
There’s no shortage of information on the topic of testing antipatterns around the web. At a high level though I’d suggest you pause if you feel like you’re writing tests that offer no value beyond ticking a code coverage box.
Some of the more common testing antipatterns involve writing tests that do little but exercise a framework or system library. For example, constructing an elaborate mock in order to verify when you call a method it returns the mock is not particularly useful.
Unless you have very good reason to do so, just let the framework and system library teams test their own code. We have more important things to do and how soul-crushing is it to write such things anyway?
Monitoring
This may be a little controversial, but monitoring is a vital component in system accuracy and is often just as valuable as upfront testing. Years of experience has taught me that we simply can’t predict all the stressors acting upon our applications until they are live in Production. Aiming for 100% code coverage does nothing more than give us a false sense of security.
We inhabit a world of Fat Tails and Black Swans in the software domain. It’s important to have enough humility to accept that we can’t predict everything that could go wrong with our applications.
It’s much better to accept that there will be situations in Production that need to be fixed. And prepare for them so we can respond quickly when the inevitable occurs. Monitoring provides the means to this end.
Summary
Striving for 100% code coverage is simply not a worthy goal in and of itself. Unless of course you’re working on an extreme edge case project. We’re much better off identifying the most likely points of failure in our application and focusing our testing efforts in this area.
We save ourselves and our clients a lot of time and money by not writing useless tests. Tests that do little but verify a framework or system library does what it’s supposed to do. Instead, plan ahead and implement a good monitoring system to quickly respond to unanticipated Production issues.