Flutter testing: A very good guide [10 Insights]

Testing is an important aspect of app development. Since Flutter provides so many types of tests (unit, widget, etc.), it can be hard to know where to begin. We’re writing this very good guide to provide an overview of our Flutter testing process and strategy at Very Good Ventures. 

We work with many different companies of varying sizes and industries. This allows us to take all of the learnings from different projects and use them to continually define and refine our engineering process. Testing is one key area that is critical to our mission of delivering high quality scalable apps. 

We’ve learned that tests can be a powerful tool to reduce risk, increase confidence in any codebase, and keep development teams on the same page by testing current expectations and assumptions. Testing is crucial for us to ensure that the code we write now will still function properly no matter how many features we add or how many developers we onboard to the project. 

Scroll on to read 10 insights about why Flutter testing is important and resources for those looking to strengthen their testing strategy.

1. Learn about all of the Flutter test options available!

In basic terms, a test is a piece of code that verifies the intended behavior of another piece of code. It can be as simple as testing that the output of a function or operation matches the expectation. For example, here is a simple test: 

import 'package:test/test.dart';
void main() {
  test('1 + 1 = 2', () {
    const expected = 2;
    final actual = 1 + 1;

    expect(actual, equals(expected));
  });
}

When it comes to building Flutter applications, there are many kinds of tests that each have their own use cases, so we created this handy chart to provide a quick overview: 

‍

2. There is a perfect formula for writing a good test: set up + side effect + expectations

A good test will always follow this same structure and test one discrete thing. If written correctly, for any given input, a test will always produce the same output. Overall, writing a test should be a predictable and arguably boring process (check out our blog on boring codebases here).

Here are some signs to know you’re writing a bad test:

• The goal of the test is unclear
• The test has more than one reason to fail (tests should focus on one discrete behavior)
• The test contains artificial delays
• The test has hidden or internal dependencies
• There are no assertions or expectations
• The order in which a test is executed affects whether it will pass

‍

3. Aim for 100% test coverage.

There are many debates about the importance of test coverage and what the ideal threshold should be. We understand that code coverage is not a perfect metric; just because you have 100% test coverage does not mean your code is 100% bug-free or 100% perfect or works 100% of the time or that tests cover 100% of possible scenarios. It simply means that 100% of lines of code are exercised by one or more tests. This also does not mean that every single line of code needs a purposeful test — something as simple as a getter or setter function may be covered by another test. 

Consistently enforcing a 100% coverage threshold is extremely important when it comes to building applications because it ensures that any future code added will 1. Require a test and 2. Won’t break existing tests. This doesn’t solve all problems, but can give some level of confidence to the quality of the codebase. 100% test coverage can also ensure that all logical branches within the codebase are accounted for, and thus, provide insight into developer intent — whether it’s a teammate later that day, you in a couple of months, or a client years down the line. 

We created the GitHub Action Very Good Coverage specifically for the purpose of measuring and enforcing 100% code coverage. We also want to note that we exclude generated code from our coverage calculation, because our primary concern is testing code that our own engineers have written. Generated code should be tested independently as part of the package’s test suite. As a side effect, code coverage is also a good indication of unused code — oftentimes, code coverage reports will show methods or classes that are unused and can be removed. 

‍

4. Tests are crucial for development teams.

Tests empower teams to build with confidence and can help them grow and scale. With every pull request, contributors should include tests to demonstrate that their contributions will not affect others’ code. This keeps everyone on the team accountable and has the added value of providing confidence that you’re not breaking something when contributing to the codebase. 

Writing tests with each pull request makes it every developer’s responsibility to maintain code quality, instead of placing the burden on the developers in charge of testing. Tests can boost productivity, teach good habits, and enforce predictable behaviors — which is crucial for companies looking to scale their apps. 

‍

5. Automated tests should be used in addition to manual testing. 

Automated tests don’t solve all of the problems — there are always some behaviors or integrations which are best tested with real world users on physical devices. For these cases, a combination of manual tests as well as automated integration tests are extremely valuable. Together they can further increase confidence that code is working correctly for the product. 

Manual tests can be expensive and time-consuming, which is why they should be used sparingly and strategically. For teams that start off exclusively relying on manual testing, the product can evolve and reach a critical point where manual testing can take weeks, sometimes without a high level of confidence. It might make sense to bring in manual testing after a code freeze and just before a release. Before deciding to rely on manual testing, you should always ask the question: can this be automated? If so, how much effort would that be? Automating tests pays off over time and can reduce the reliance on manual testing down the line, saving time, stress, and improving the overall quality of the product. 

‍

6. Automated tests should be baked into the development process.

Some developers may avoid writing tests because they take additional time to write and maintain. They may even be the first thing to go when a developer or team wants to move fast. However, investing time in writing tests now can be critical in relieving pressure and eliminating the last minute scramble to fix a showstopper bug before shipping an app. 

This is why we advocate for writing tests alongside writing features — not as a completely separate effort when preparing to ship an app — the more you get into the habit of testing as you go, the more confidence you will have in the quality of your codebase. Ideally your product should be shippable at any point and tests can help you get there. 

There has never been a better time to start testing than right now! As with learning anything new, there may be a learning curve if you’re just starting to bolster your test coverage, but with practice, writing tests will become a natural extension of the development process. You will soon find that writing tests is a positive feedback loop. Start by establishing a current baseline, whether that’s 10% or 80% and try to increase gradually over time, while preventing regressions. A tool like Very Good Coverage can help you measure and stick to this threshold. Once you reach 100% coverage, your work is not done! You can continue to look for opportunities to test edge cases and more complex scenarios. In addition, any time you find a bug, the fix should always be accompanied by one or more tests to ensure the bug never surfaces again.

7. Writing tests can make the code review process easier.

Internally, tests can greatly help with manual code reviews. When working on a team, everyone should be writing tests and submitting them with PRs (again, tests should be baked into the development process, not written later). This is especially beneficial for new teammates or junior teammates who are starting to familiarize themselves with the codebase and will likely be the future maintainers of the project. Beyond setting a good example, having comprehensive test coverage can serve as an educational resource because tests should communicate intent. Scanning tests for a feature or specific part of a codebase oftentimes gives a good overview of what a feature does and how it’s intended to function. Since it’s all code, at the end of the day, teammates can use their IDE of choice to tinker with implementation and see how their changes impact the tests.

If possible, we recommend assigning two (or more!) designated code owners to a project so that they can be automatically added as reviewers before code gets merged. You can even get more granular and assign code owners for specific subdirectories of a project. This way, if someone on your team is an expert or a designated leader of a specific part of the codebase, they can automatically be added as a reviewer when changes occur to it.

Over time, as testing becomes ingrained into team process and culture, PR review becomes much more predictable, reduces back-and-forth between reviewers and the author, and overall speeds up the development velocity.‍

‍

8. Tests should only fail when there is a bug in the code or a product requirement changes.

Automated testing can reduce pressure on developers and place the pressure on the system and the team. A good mentality to have is: if a bug is introduced, it’s not an individual’s fault, it’s the system’s fault. To account for it, the system (automated test suites and any additional tooling such as linter, formatters, performance benchmarks, etc.) should be updated or enhanced to ensure that the same bug never happens again. 

If a requirement changes, developers will need to update tests accordingly. Developers should feel empowered to discuss requirements when tests fail. For example, if a test fails, you should ask yourself: did I truly introduce a bug, or is the requirement unclear, or can the test be improved? Sometimes, this is a sign that planning was imperfect or the testing infrastructure has gaps. In any case, it’s a moment to reflect and decide the best path forward as a team. Without tests, these moments of reflection might otherwise not happen and could further decrease quality of the code as well as introduce ambiguity in requirements and intended behavior. Ultimately it’s important to have individuals feel empowered and responsible to deliver a high quality product, rather than offloading that responsibility to a dedicated testing team.

‍

9. Tests save companies time and money.

If writing tests is extremely time-consuming or difficult, that could be a sign that the underlying implementation is either poorly architected or implemented. It’s important to identify this as early as possible and address the root cause of the problem before building on top of a shaky foundation. An untested codebase simply cannot scale and may also mean more bugs and problems in the future.

This is why tests are extremely important to write as you’re building your codebase. They can save teams time and money; from our experience, one of the reasons for deciding to do a rewrite is simply lack of confidence and stability due to limited or no tests. It can be more expensive and time-consuming to rewrite a project from scratch than it would be to incorporate testing as part of the development process.

‍

10. Never stop learning! There are many great resources for writing tests.

If you want to learn more about Flutter testing, we recommend the following:

• Our teammate Jorge Coca developed an amazing comprehensive course called Testing Fundamentals of Flutter. Originally available on Caster.IO, the full course is now available in this YouTube playlist.
• Another great resource is Reso Coder’s video series on Test-Driven Development.
• Finally, if you use our teammate Felix Angelov’s flutter_bloc library for state management, check out bloc_test, a Dart package that makes testing blocs and cubits easy. 

‍

Felix Angelov and Jorge Coca contributed to this article.