Road to 100% Test Coverage

At Very Good Ventures, we incorporate the “Very Good” concept into our services like Very Good News and Very Good Start and I would like to extend it to testing. This isn't just a nod to our company’s name; it underscores our belief in the significance of testing quality over quantity, steering away from a mere focus on metrics. This is the main topic I talked about when I had the opportunity to be a speaker at Fluttercon Berlin and Flutterconf Latam this year. 

Let's delve into some strategies and insights we employ at VGV for effective testing.

Why (Very Good) Tests Are Important

The main question you might ask if you are new to testing is “Why are tests important? Is it not enough to go through my app manually and verify there are no bugs?”. Hopefully, I can answer those in this blog post for you. But the first thing you need to know about testing is that you will need a process to ensure that your app quality is gonna be increased: It’s not just about adding a final step in your app development phase. If you start changing your mindset about testing you will see yourself avoiding stressful situations and you will start enjoying the path. How? Let me highlight three things -among others-  that we at VGV consider important about testing:

‍

1. It makes you think about your architecture. Most of the time a hard test is linked to poor architecture. So, the more you test, the more you will feel confident about your code architecture. But we will get beyond this point further in this blog. 
2. 100% test coverage does not mean 0% bugs. You shouldn’t focus on the number itself, but on covering all possible scenarios.
   Let me share a very good tip: If for some reason a case was not covered and a bug is found, make sure to consistently add a test for that case to prevent its recurrence.
3. The more you test, the more you will control all your business logic. This is especially important when reviewing a new code base to see if it has tests. If not, it's probably a good time to add them while onboarding in the code.

Your final goal should be to increase confidence in any future change in the code to deliver fast and safely.

Architecture Matters

As mentioned before, one key point in order to have a smooth approach to testing is having a good architecture. Especially for developers new to the practice, they might experience a big learning curve for testing, but in reality, most of the problems they will face with it will be related to bad architectural decisions. 

‍

This blog post is not intended to be focused on architecture, but here are some pieces of advice that will help with testing:

‍

• Split your code into layers that allow dependency injection.
• Don't over complicate your classes and methods (SOLID)
• Don't mix layers. For example: Don't put business logic into your UI.
• Use libraries/packages that help you with testing. For example: Static methods are known for being quite complex to test.

If you need more insights into how to build a good and scalable architecture for testing, you can take a look at this blog post. 

But, What Is Code Coverage? 

Code coverage is a metric that allows you to identify which parts of your code have been exercised by a test. In Dart/Flutter, we measure that by lines of code.

‍

In order to collect the coverage we need to follow 2 simple steps:

‍

1. Execute our tests collecting coverage

very_good test --test-randomize-ordering-seed random --coverage

As you might have noticed, we execute the tests in random order to verify we don't have side effects between those.

‍

2. Visualize the coverage report

genhtml ./coverage/lcov.info -o coverage && open ./coverage/index.html

Why 100% and Not 80%?

As I mentioned before, when you are doing tests, the last thing you should be thinking about is the metric or the number itself. You can have 100% coverage and it still is possible that your tests are not really valuable. Code coverage is only a metric, but If we could add a restriction to ensure that we cover all possible scenarios we would use that one instead.

‍

For example, consider a scenario where you have a button adding a bloc event upon clicking on it, while you may have a test that clicks on the button, it's crucial to ensure that the test verifies the execution of the underlying logic as well, because you probably have covered this button tap but the reality is that you are not testing anything. 

‍

See the example of covering the same code but not the same scenario below:

At the same time, if you start defining a number lower than 100%, you need to start making the hard decision on what areas you will not be covering. Should you skip models? Should you skip UI? Should you skip external library usage?

‍

People are surprised when we aim at VGV for 100% code coverage, and a recurrent question that was asked to me in both talks was about exceptions.  We only have a few exceptions, and most of them are related to auto generated code, for example, localized strings or assets since there is no added value about testing programmatically on those.

Summing-up

At Very Good Ventures, we have been putting all emphasis on prioritizing test quality over metrics, aiming for 100% test coverage, with a focus on comprehensive testing that ideally covers all scenarios. Emphasizing architectural best practices such as layer separation and simplicity, plays a crucial role while aiming to bring confidence in every code change, ensuring fast and secure delivery through meticulous testing practices.

‍

Feel free to reach out to me on my LinkedIn or Twitter account to learn more!