# Supercharge your Flutter tests with Very Good CLI

> A closer look at the very_good test command

- Source: https://verygood.ventures/blog/flutter-tests-very-good-cli/
- Published: 2022-03-25
- Author: Felix Angelov
- Tags: Best Practices, Testing

---

Last week we made a number of exciting updates to [Very Good CLI](https://pub.dev/packages/very_good_cli) focused on improvements to testing Dart and Flutter apps 🎉.

[Very Good CLI v0.6.0](https://github.com/VeryGoodOpenSource/very_good_cli/releases/tag/v0.6.0) now features a new test command that provides additional functionality on top of the existing command line tooling for [**dart test**](https://dart.dev/tools/dart-test) and [**flutter test**](https://docs.flutter.dev/reference/flutter-cli).

Included are performance optimizations, improvements to the developer experience when measuring and enforcing tests coverage metrics, and improvements when working with multimodule monorepos.

## Overview 🚀

Very Good CLI's new **test** command includes the following flags and arguments. You can always see the most up-to-date list by running **very\_good test --help**.

```shell
$ very_good test --help
Run tests in a Dart or Flutter project.
Usage: very_good test [arguments]
-h, --help                            Print this usage information.
    --coverage                        Whether to collect coverage information.
-r, --recursive                       Run tests recursively for all nested packages.
    --[no-]optimization               Whether to apply optimizations for test performance.
                                      (defaults to on)
    --exclude-coverage                A glob which will be used to exclude files that match from the coverage.
-x, --exclude-tags                    Run only tests that do not have the specified tags.
    --min-coverage                    Whether to enforce a minimum coverage percentage.
    --test-randomize-ordering-seed    The seed to randomize the execution order of test cases within test files.
Run "very_good help" to see global options.
```

## How its made 🛠

In order to build a custom test command, we first needed a way to programatically run tests that would fit nicely with our existing tooling for Very Good CLI. As a result, we created the [Very Good Test Runner](https://pub.dev/packages/very_good_test_runner) package which provides APIs to run Dart and Flutter tests programatically thanks to the [JSON Reporter Protocol](https://github.com/dart-lang/test/blob/master/pkgs/test/doc/json_reporter.md).

Internally, **very\_good\_test\_runner** uses the [Process class](https://api.dart.dev/stable/2.16.2/dart-io/Process-class.html) to spawn **dart test** and **flutter test** processes respectively. We specify the **\--reporter=json** option to receive a machine-readable representation of the underlying test runner's progress. **very\_good\_test\_runner** exposes a stream of **TestEvent** objects which consumers can subscribe to in order to build custom testing tooling.

```dart
import 'package:very_good_test_runner/very_good_test_runner.dart';

void main() {
  const arguments = ['--coverage'];
  const workingDirectory = 'path/to/project';

  // Run Dart tests and collect coverage in the working directory.
  dartTest(
    arguments: arguments,
    workingDirectory: workingDirectory,
  ).listen((TestEvent event) {
    // React to `TestEvent` instances.
  });

  // Run Flutter tests and collect coverage in the working directory.
  flutterTest(
    arguments: arguments,
    workingDirectory: workingDirectory,
  ).listen((TestEvent event) {
    // React to `TestEvent` instances.
  });
}
```

Custom testing tools can enable you to customize how test results are displayed, detect tests that are abnormally slow, re-run only the tests that have failed from previous test runs, and more. Another great example of a custom testing tool is the [spec_cli](https://pub.dev/packages/spec_cli) from [Invertase](https://github.com/invertase). The possibilities for custom test tooling are endless and we're very excited to see what other tools the community builds.

## Testing Improvements ✨

### Performance Optimizations

Very Good CLI takes advantage of a [known performance optimization](https://github.com/flutter/flutter/issues/86722#issuecomment-914688482) to decrease test run times. From our experience, this is particularly significant in large codebases when collecting coverage metrics. As the number of tests increases, the cost of running the tests and collecting coverage metrics also increases so applying this optimization can reduce test runs quite dramatically (see below comparison).

![Test run time comparison without Very Good CLI optimization](/assets/images/blog/flutter-tests-very-good-cli/body-0.png)

![Test run time comparison with Very Good CLI optimization enabled](/assets/images/blog/flutter-tests-very-good-cli/body-1.png)

This additional optimization step is performed by default but can be disabled with the **\--no-optimization** flag. You may want to disable optimizations in order to improve the traceability of failing tests or for smaller codebases with fewer tests to reduce the initial overhead of performing the optimization step.

```shell
# run tests with performance optimizations
very_good test

# run tests without performance optimizations
very_good test --no-optimization
```

As part of the performance optimization step, Very Good CLI uses a [mason](https://github.com/felangel/mason) brick to aggregate all tests and generate a single **.test\_runner.dart** file.

```dart
// GENERATED CODE - DO NOT MODIFY BY HAND
// Consider adding this file to your .gitignore.
import 'package:test/test.dart';
import 'app/view/app_test.dart' as app_view_app_test_dart;
import 'counter/cubit/counter_cubit_test.dart' as counter_cubit_counter_cubit_test_dart;
import 'counter/view/counter_page_test.dart' as counter_view_counter_page_test_dart;

void main() {
  group('app_view_app_test_dart', app_view_app_test_dart.main);
  group('counter_cubit_counter_cubit_test_dart', counter_cubit_counter_cubit_test_dart.main);
  group('counter_view_counter_page_test_dart', counter_view_counter_page_test_dart.main);
}
```

Then, the test execution consists of executing the single **.test\_runner.dart** file:

```dart
final generator = await MasonGenerator.fromBundle(testRunnerBundle);
...
await generator.generate(
  target,
  vars: vars,
  fileConflictResolution: FileConflictResolution.overwrite,
);
...
flutterTest(
  workingDirectory: workingDirectory,
  arguments: [
    ...
    if (optimizePerformance) p.join('test', '.test_runner.dart')
  ],
).listen((event) {...});
```

### Enforcing Coverage

The new **test** command also provides an option to specify a minimum test coverage threshold. If the reported coverage drops below that threshold, the test run will exit with an error code.

```shell
# Run tests, collect coverage, and enforce 100% coverage
very_good test --coverage --min-coverage 100
```

This is particularly useful when running tests as part of your continuous integration because you can fail a build if test coverage drops below the accepted threshold (we recommend 100% 💯).

### Excluding Coverage

In some cases, it's handy to be able to exclude certain files from test coverage.

For example, you may want to exclude code generated by [**build_runner**](https://pub.dev/packages/build_runner) from test coverage. The **\--exclude-coverage** option allows you to specify a [glob](https://pub.dev/packages/glob) which will be used to exclude files from coverage.

```shell
# Run tests, collect coverage, exclude generated dart files from coverage, and enforce 100% coverage
very_good test --coverage --exclude-coverage "*.g.dart" --min-coverage 100
```

### Run Recursively

When working on a monorepo with many packages, it can be quite tedious to run tests in all sub-packages. For example, after a large refactor you may want to verify that no regressions were introduced. Previously, you'd have to manually run the tests in each affected package one at a time. With the new **test** command in Very Good CLI, it's easy to run tests in all nested packages via the **\--recursive** flag (**\-r** for short).

```shell
# Run all tests in the current package as well as nested packages.
very_good test --recursive
```

❗️ Currently, the **very\_good test** command must be executed from the root directory of a package so be sure to keep that in mind when using the **\--recursive** flag.

## Summary

Very Good CLI now ships with some new testing capabilities that we hope will make running tests in Flutter/Dart even more enjoyable. We're excited to continue to expand the capabilities of Very Good CLI and plan to incorporate the testing improvements into [very_good_workflows](https://github.com/VeryGoodOpenSource/very_good_workflows) shortly.

Until next time, happy testing 🧪🦄!