LogoPatrol

Getting started

Check out our video version of this tutorial on YouTube!

If you want to use Patrol finders in your existing widget or golden tests, go to Using Patrol finders in widget tests.

In this tutorial, we are using example app, which has package name com.example.myapp on Android, bundle id com.example.MyApp on iOS, com.example.macos.MyApp on macOS and My App name on all platforms. Replace any occurences of those names with proper values.

Support for macOS is in alpha stage. Please be aware that some features may not work as expected. There is also no native automation support for macOS yet. If you encounter any issues, please report them on GitHub.

Add dependency on patrol#

If you haven't already, add a dependency on the patrol package in the dev_dependencies section of pubspec.yaml. patrol package requires Android SDK version 21 or higher.

flutter pub add patrol --dev

Configure Patrol in pubspec.yaml#

Create patrol section in your pubspec.yaml:

pubspec.yaml
dependencies:
  # ...

dev_dependencies:
  # ...

patrol:
  app_name: My App
  android:
    package_name: com.example.myapp
  ios:
    bundle_id: com.example.MyApp
  macos:
    bundle_id: com.example.macos.MyApp

If you don't know where to get package_name and bundle_id from, see the FAQ section.

Install patrol_cli#

Patrol CLI (command-line interface) is a small program that enables running Patrol UI tests. It is necessary to run UI tests (flutter test won't work! Here's why).

  1. Install patrol_cli executable:

    dart pub global activate patrol_cli
    

Make sure to add patrol to your PATH environment variable. It's explained how to do it in the README.

  1. Verify that installation was successful and your environment is set up properly:

    patrol doctor
    

    Example output:

    Patrol CLI version: 2.3.1+1
    Android:
    • Program adb found in /Users/username/Library/Android/sdk/platform-tools/adb
    • Env var $ANDROID_HOME set to /Users/username/Library/Android/sdk
    iOS / macOS:
    • Program xcodebuild found in /usr/bin/xcodebuild
    • Program ideviceinstaller found in /opt/homebrew/bin/ideviceinstaller
    

    Be sure that for the platform you want to run the test on, all the checks are green.

Patrol CLI invokes the Flutter CLI for certain commands. To override the command used, pass the --flutter-command argument or set the PATROL_FLUTTER_COMMAND environment variable. This supports FVM (by setting the value to fvm flutter), puro (puro flutter) and potentially other version managers.

Integrate with native side#

The 3 first steps were common across platforms. The rest is platform-specific.

Psst... Android is a bit easier to set up, so we recommend starting with it!

Android setup
  1. Go to android/app/src/androidTest/java/com/example/myapp/ in your project directory. If there are no such folders, create them. Remember to replace /com/example/myapp/ with the path created by your app's package name.

  2. Create a file named MainActivityTest.java and copy there the code below.

MainActivityTest.java
package com.example.myapp; // replace "com.example.myapp" with your app's package

import androidx.test.platform.app.InstrumentationRegistry;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.junit.runners.Parameterized;
import org.junit.runners.Parameterized.Parameters;
import pl.leancode.patrol.PatrolJUnitRunner;

@RunWith(Parameterized.class)
public class MainActivityTest {
    @Parameters(name = "{0}")
    public static Object[] testCases() {
        PatrolJUnitRunner instrumentation = (PatrolJUnitRunner) InstrumentationRegistry.getInstrumentation();
        // replace "MainActivity.class" with "io.flutter.embedding.android.FlutterActivity.class" 
        // if your AndroidManifest is using: android:name="io.flutter.embedding.android.FlutterActivity"
        instrumentation.setUp(MainActivity.class);
        instrumentation.waitForPatrolAppService();
        return instrumentation.listDartTests();
    }

    public MainActivityTest(String dartTestName) {
        this.dartTestName = dartTestName;
    }

    private final String dartTestName;

    @Test
    public void runDartTest() {
        PatrolJUnitRunner instrumentation = (PatrolJUnitRunner) InstrumentationRegistry.getInstrumentation();
        instrumentation.runDartTest(dartTestName);
    }
}
  1. Go to the build.gradle file, located in android/app folder in your project directory.

  2. Add these 2 lines to the defaultConfig section:

  testInstrumentationRunner "pl.leancode.patrol.PatrolJUnitRunner"
  testInstrumentationRunnerArguments clearPackageData: "true"
  1. Add this section to the android section:
  testOptions {
    execution "ANDROIDX_TEST_ORCHESTRATOR"
  }
  1. Add this line to dependencies section:
  androidTestUtil "androidx.test:orchestrator:1.4.2"
iOS setup
  1. Open ios/Runner.xcworkspace in Xcode.

  2. Create a test target if you do not already have one (see the screenshot below for the reference). Select File > New > Target... and select UI Testing Bundle. Change the Product Name to RunnerUITests. Set the Organization Identifier to be the same as for the Runner (no matter if you app has flavors or not). For our example app, it's com.example.MyApp just as in the pubspec.yaml file. Make sure Target to be Tested is set to Runner and language is set to Objective-C. Select Finish.

Xcode iOS test target

  1. 2 files are created: RunnerUITests.m and RunnerUITestsLaunchTests.m. Delete RunnerUITestsLaunchTests.m through Xcode by clicking on it and selecting Move to Trash.

  2. Make sure that the iOS Deployment Target of RunnerUITests within the Build Settings section is the same as Runner. The minimum supported iOS Deployment Target is 11.0. For the example app, we set it to 13.0 because it's required by the app dependencies.

Xcode iOS deployment target

Xcode iOS deployment target 2

  1. Replace contents of RunnerUITests.m file with the following:
ios/RunnerUITests/RunnerUITests.m
@import XCTest;
@import patrol;
@import ObjectiveC.runtime;

PATROL_INTEGRATION_TEST_IOS_RUNNER(RunnerUITests)

Add the newly created target to ios/Podfile by embedding in the existing Runner target.

ios/Podfile
target 'Runner' do
  # Do not change existing lines.
  ...

  target 'RunnerUITests' do
    inherit! :complete
  end
end
  1. Create an empty file integration_test/example_test.dart in the root of your Flutter project. From the command line, run the following command and make sure it completes with no errors:
$ flutter build ios --config-only integration_test/example_test.dart
  1. Go to your ios directory and run:
$ pod install --repo-update
  1. Open your Xcode project and Make sure that for each build configuration, the RunnerUITests have the same Configuration Set selected as the Runner:

Xcode config setup

  1. Go to RunnerUITests -> Build Phases and add 2 new "Run Script Phase" Build Phases. Name them xcode_backend build and xcode_backend embed_and_thin.

Xcode config setup

  1. Arrange the newly created Build Phases in the order shown in the screenshot below.

Xcode config setup

  1. Paste this code into the xcode_backend build Build Phase:
/bin/sh "$FLUTTER_ROOT/packages/flutter_tools/bin/xcode_backend.sh" build
  1. Paste this code into the xcode_backend embed_and_thin Build Phase:
/bin/sh "$FLUTTER_ROOT/packages/flutter_tools/bin/xcode_backend.sh" embed_and_thin
  1. Xcode by default also enables a "parallel execution" setting, which breaks Patrol. Disable it for all schemes (if you have more than one):
  1. Go to RunnerUITests -> Build Settings, search for User Script Sandboxing and make sure it's set to No.
macOS setup
  1. Open macos/Runner.xcworkspace in Xcode.

  2. Create a test target if you do not already have one via File > New > Target... and select UI Testing Bundle. Change the Product Name to RunnerUITests. Make sure Target to be Tested is set to Runner and language is set to Objective-C. Select Finish.

  3. 2 files are created: RunnerUITests.m and RunnerUITestsLaunchTests.m. Delete RunnerUITestsLaunchTests.m through Xcode.

  4. Make sure that the macOS Deployment Target of RunnerUITests within the Build Settings section is the same as Runner. The minimum supported macOS Deployment Target is 10.14.

Xcode macOS deployment target

Xcode macOS deployment target 2

  1. Replace contents of RunnerUITests.m file with the following:
macos/RunnerUITests/RunnerUITests.m
@import XCTest;
@import patrol;
@import ObjectiveC.runtime;

PATROL_INTEGRATION_TEST_MACOS_RUNNER(RunnerUITests)

Add the newly created target to macos/Podfile by embedding in the existing Runner target.

macos/Podfile
target 'Runner' do
  # Do not change existing lines.
  ...

  target 'RunnerUITests' do
    inherit! :complete
  end
end
  1. Create an empty file integration_test/example_test.dart in the root of your Flutter project. From the command line, run:
$ flutter build macos --config-only integration_test/example_test.dart
  1. Go to your macos directory and run:
$ pod install --repo-update
  1. Go to RunnerUITests -> Build Phases and add 2 new "Run Script Phase" Build Phases. Rename them to xcode_backend build and xcode_backend embed_and_thin by double clicking on their names.

Xcode config setup

  1. Arrange the newly created Build Phases in the order shown in the screenshot below.

Xcode config setup

  1. Paste this code into the first macos_assemble build Build Phase:
/bin/sh "$FLUTTER_ROOT/packages/flutter_tools/bin/macos_assemble.sh" build
  1. Paste this code into the second macos_assemble embed Build Phase:
/bin/sh "$FLUTTER_ROOT/packages/flutter_tools/bin/macos_assemble.sh" embed
  1. Xcode by default also enables a "parallel execution" setting, which breaks Patrol. Disable it for all schemes (if you have more than one):
  1. Go to RunnerUITests -> Build Settings, search for User Script Sandboxing and make sure it's set to No.

  2. Go to Runner -> Signing & Capabilities. Make sure that in all App Sandbox sections, Incoming Connections (Server) and Outgoing Connections (Client) checkboxes are checked.

Xcode entitlements setup

  1. Copy DebugProfile.entitlements and Release.entitlements files from macos/Runner to macos/RunnerUITests directory.

  2. Go to RunnerUITests -> Build Settings and set Code Signing Entitlements to RunnerUITests/DebugProfile.entitlements for Debug and Profile configuration and to RunnerUITests/Release.entitlements for Release configuration.

Xcode RunnerUITests entitlements setup

Create a simple integration test#

Let's create a dummy Flutter integration test that you'll use to verify that Patrol is correctly set up.

Paste the following code into integration_test/example_test.dart:

integration_test/example_test.dart
import 'dart:io';

import 'package:flutter/material.dart';
import 'package:flutter_test/flutter_test.dart';
import 'package:patrol/patrol.dart';

void main() {
  patrolTest(
    'counter state is the same after going to home and switching apps',
    ($) async {
      // Replace later with your app's main widget
      await $.pumpWidgetAndSettle(
        MaterialApp(
          home: Scaffold(
            appBar: AppBar(title: const Text('app')),
            backgroundColor: Colors.blue,
          ),
        ),
      );

      expect($('app'), findsOneWidget);
      if (!Platform.isMacOS) {
        await $.native.pressHome();
      }
    },
  );
}

It does only 2 things:

  • first, it finds a text app
  • then (on mobile platforms), it exits to home screen

It's a very simple test, but it's enough to verify that Patrol is correctly set up. To run integration_test/example_test.dart on a connected Android, iOS or macOS device:

patrol test -t integration_test/example_test.dart

If the setup is successful, you should see a TEST PASSED message. If something went wrong, please proceed to the FAQ section which might contain an answer to your issue.

If your app is using flavors, then you can pass them like so:

patrol test --target integration_test/example_test.dart --flavor development

or you can specify them in pubspec.yaml (recommended):

pubspec.yaml
patrol:
  app_name: My App
  flavor: development
  android:
    package_name: com.example.myapp
  ios:
    bundle_id: com.example.MyApp
    app_name: The Awesome App
  macos:
    bundle_id: com.example.macos.MyApp

To prevent issues during Patrol tests, please follow these guidelines:

  1. Do not call IntegrationTestWidgetsFlutterBinding.ensureInitialized. Patrol automatically initializes its own test binding.
  2. Do not modify the global FlutterError.onError callback. Patrol's internals depend on it. Keep in mind that this callback can also be modified by popular packages such as Sentry or Crashlytics. In such cases, you can disable them for Patrol tests.

If you are looking for a working example of a Flutter app with Patrol tests, check out the example app in the patrol repository.

FAQ#

Testing fails with error inside 'integration_test/test_bundle.dart'

The reason is probably a mismatch of patrol and patrol_cli versions. Go to Compatibility table and make sure that the versions of patrol and patrol_cli you are using are compatible.

'example_test.dart' passed but I can't get my application to run within the patrol test.

To run your application within the patrol test, you need to call $.pumpWidgetAndSettle(), and pass your application's main widget to it. Be sure that you registered all the necessary services before calling $.pumpWidgetAndSettle(). Here's the example of running an app within the patrol test:


void main() {
  patrolTest('real app test', ($) async {

  // Do all the necessary setup here (DI, services, etc.)

  await $.pumpWidgetAndSettle(const MyApp()); // Your's app main widget

  // Start testing your app here

  });
}

It's a good practice to create a setup wrapper function for your tests, so you don't have to repeat the same code in every test. Look at the example of a wrapper function.

Android#

Where do I get 'package_name' from?

Go to android/app/build.gradle and look for applicationId in defaultConfig section.

Running patrol test fails containing 'Unsupported class file major version' error

It's most likely caused by using incompatible JDK version. Run javac -version to check your JDK version. Patrol officially works on JDK 17, so unexpected errors may occur on other versions. If you have AndroidStudio or Intellij IDEA installed, you can find the path to JDK by opening your project's android directory in AS/Intellij and going to Settings -> Build, Execution, Deployment -> Build Tools -> Gradle -> Gradle JDK. Learn more

iOS#

Where do I get `bundle_id` from?

For iOS go to ios/Runner.xcodeproj/project.pbxproj and look for PRODUCT_BUNDLE_IDENTIFIER. For macOS go to macos/Runner.xcodeproj/project.pbxproj and look for PRODUCT_BUNDLE_IDENTIFIER.

When I run tests the simulator is getting cloned

Make sure that you disabled "Paralell execution" for all schemes in Xcode. See this video for details.

Running test stops on 'Wait for com.example.myapp to idle'

Open your XCode project, go to Runner target -> Build Settings, search for FLUTTER_TARGET and remove set path to targets for all configurations. Alternatively, you can search for a FLUTTER_TARGET in your project files and remove it (both value and key) from *.xcconfig and *.pbxproj files.

When running a test, real app without test is opened instead

Open your XCode project, go to Runner target -> Build Settings, search for FLUTTER_TARGET and remove set path to targets for all configurations. Alternatively, you can search for a FLUTTER_TARGET in your project files and remove it (both value and key) from *.xcconfig and *.pbxproj files.

Build is failing with errors on mismatching iOS deployment versions

Check if this line in Podfile is present and uncommented.

platform :ios, '11.0'

If yes, then check if iOS deployment version in Xcode project's Build Settings section for all targets (Runner and RunnerUITests) are set to the same value as in Podfile (in case presented in snippet above, all should be set to 11.0).

If you couldn't find an answer to your question/problem, feel free to ask on Patrol Discord Server.

Going from here#

To learn how to write Patrol tests, see finders and native automation sections.