test

Synopsis

Run integration tests.

patrol test

To see all available options and flags, run patrol test --help.

Description

This command is the one use you'll be using most often.

patrol test does the following things:

1. Builds the app under test (AUT) and the instrumentation app
2. Installs the AUT and the instrumentation on the selected device
3. Runs the tests natively, and reports results back in native format.

Under the hood, it calls Gradle (when testing on Android) and xcodebuild (when testing on iOS).

Discussion

By default, patrol test runs all integration tests (files ending with _test.dart located in the patrol_test directory). You can customize the test directory by setting test_directory in your pubspec.yaml under the patrol section.

To run a single test, use --target:

patrol test --target patrol_test/login_test.dart

You can use --target more than once to run multiple tests:

patrol test \
  --target patrol_test/login_test.dart \
  --target patrol_test/app_test.dart

Or alternatively:

patrol test --targets patrol_test/login_test.dart,patrol_test/app_test.dart

Test files must end with _test.dart. Otherwise the file is not considered a test and is not run.

Tags

You can use tags to run only tests with specific tags.

First specify tags in your patrol tests:

  patrol(
    'example test with tag',
    tags: ['android'],
    ($) async {
      await createApp($);

      await $(FloatingActionButton).tap();
      expect($(#counterText).text, '1');
    },
  );

  patrol(
    'example test with two tags',
    tags: ['android', 'ios'],
    ($) async {
      await createApp($);

      await $(FloatingActionButton).tap();
      expect($(#counterText).text, '1');
    },
  );

Then you can run tests with the tags you specified:

patrol test --tags android
patrol test --tags=android
patrol test --tags='android||ios'
patrol test --tags='(android || ios)'
patrol test --tags='(android && tablet)'

You can also use --exclude-tags to exclude tests with specific tags:

patrol test --exclude-tags android
patrol test --exclude-tags='(android||ios)'

Coverage

To collect coverage from patrol tests, use --coverage.

patrol test --coverage

The LCOV report will be saved to /coverage/patrol_lcov.info.

Additionally, you can exclude certain files from the report using glob patterns and --coverage-ignore option. For instance,

patrol test --coverage --coverage-ignore="**/*.g.dart"

excludes all files ending with .g.dart.

Coverage on the web

Web coverage is collected differently from other platforms, so a couple of things differ:

• Lines are reported as covered or uncovered only, without the per-line hit counts available on other platforms.
• The covered-line set may differ slightly from the same code measured on mobile.

The report format is otherwise identical.

Build versioning

You can specify custom build number and build name using the --build-name and --build-number flags. These work the same as in Flutter CLI:

• --build-name: Version name of the app. (e.g. 1.2.3)
• --build-number: Version code of the app. (e.g. 123)

patrol test --build-name=1.2.3 --build-number=123
patrol test --target patrol_test/login_test.dart --build-name=1.2.3 --build-number=123

Isolation of test runs

To achieve full isolation between test runs:

• On Android: set clearPackageData to true in your build.gradle file,
• On iOS Simulator: use the --full-isolation flag

patrol test --full-isolation

Video recording

Record a video of each test case using --record-video:

patrol test --record-video

Videos are saved as .mp4 files in <test-directory>/videos (override with --video-output-dir). Supported on Android emulators and iOS simulators.

• Physical Android devices may also work, depending on the vendor (some block screenrecord).
• Physical iOS devices are not supported.
• --video-size and --video-bit-rate apply to Android only.

Screenshots

Patrol can capture native screenshots on Android — automatically on test failure (opt-in) and on demand from a test:

• Enable failure screenshots by adding screenshot_on_failure: true to the patrol section of your pubspec.yaml.
• Capture on demand with await $.takeNativeScreenshot('tag') inside a test.

The capture happens natively (via androidx.test) at the moment of failure, before teardown, so it shows the actual failing screen. Patrol writes the PNG on the device to /sdcard/Download/screenshots/<class>/<method>/.

On patrol test, screenshots are pulled to <test-directory>/screenshots after the run (override with --screenshots-output-dir). On device farms the farm collects them:

• BrowserStack App Automate: enable the Espresso debugscreenshots: true capability; images appear in the Screenshots tab. Also enable build-time test discovery (emit_test_manifest) so each test reports a URL-safe name — BrowserStack can't render screenshot URLs with spaces/commas, which the default runDartTest[...] names contain.
• Firebase Test Lab: pass --directories-to-pull=/sdcard/Download/screenshots.

Web Platform

Patrol supports running tests on Flutter web using Playwright. To run tests on web:

patrol test --device chrome

When running on web:

• Tests execute in Chromium browser via Playwright
• CLI arguments can be used to configure Playwright
• Test results are generated in test-results/

Arguments

Playwright configuration is updated with values passed to the command. This allows direct control over Playwright features such as reporting. To see the full list of supported arguments, run patrol test --help.

Note: Some arguments are not supported on web:

• --flavor: Flavors are not supported for Flutter web
• --uninstall: Not applicable to web platform
• --clear-permissions: Not applicable to web platform
• --full-isolation: Not applicable to web platform

Running without rebuilding

When build-time test discovery is enabled (patrol.emit_test_manifest: true), already-built tests can be re-run without rebuilding using a separate command — see patrol test-without-building.

Under the hood

patrol test basically calls patrol build and then runs the built app binaries. For more info, read docs of patrol build.