# Patrol
A powerful, multiplatform E2E UI testing framework for Flutter apps that overcomes the limitations of integration\_test by handling native interactions. Developed since 2022 by [LeanCode][leancode] - top Flutter development company for Enterprise and Scale-ups - battle-tested and shaped by production-grade experience.
[
](https://leancode.co/webinar/e2e-testing-in-flutter?utm_source=patrol_page\&utm_medium=banner\&utm_campaign=webinar)
## Why choose Patrol? [#why-choose-patrol]
### Native Access, Redefined [#native-access-redefined]
Unlock [**native platform features**][native] right within your Flutter tests. With Patrol, you can:
* Interact with **permission dialogs**, **notifications**, and **WebViews**.
* Modify **device settings**, toggle **Wi-Fi**, and more.
* Achieve all this effortlessly using plain **Dart** code.
### Intuitive Test Writing [#intuitive-test-writing]
Say goodbye to complexity with Patrol’s custom finder system.
* Streamline your test code with a shorter, more readable, [**new custom finder system**][finders].
* Enjoy the speed and convenience of [**Hot Restart**][hot restart], which makes integration testing faster, easier, and more fun.
* Quickly inspect the currently visible Android/iOS views and discover their properties with the **Patrol DevTools extension**.
### Production-Ready Integration Testing [#production-ready-integration-testing]
Patrol revolutionizes Flutter’s built-in `integration_test` plugin:
* Overcomes its limitations with **full test isolation** between tests and **sharding**.
* Delivers a robust, **production-grade solution** for your app testing needs.
* Offers **console logs** to get real-time insights during test execution.
### Compatible with Device Farms [#compatible-with-device-farms]
With Patrol's native-like testing capabilities, you can use popular device farms like:
* Firebase Test Lab
* BrowserStack
* LambdaTest
* Marathon
* emulator.wtf
* AWS Device Farm
## Trusted by LeanCode and the Flutter Community [#trusted-by-leancode-and-the-flutter-community]
Patrol is a fully open-source project, and we're proud to share it with the amazing
Flutter community. Patrol isn’t just a tool; it’s a commitment to quality. At LeanCode,
we use Patrol to test production-grade apps for clients across industries, and now, you can do the same!
[Get Patrol from pub.dev now!][patrol_on_pubdev]
## Need expert help? [#need-expert-help]
LeanCode offers end-to-end automated testing services tailored for Flutter apps, as well as Patrol setup and Patrol training to help your teams get the most out of Patrol. Interested? Click on a dedicated banner below.
[
](https://leancode.co/products/automated-ui-testing-in-flutter?utm_source=patrol_page\&utm_medium=banner\&utm_campaign=service)
[
](https://leancode.co/products/patrol-setup-training?utm_source=patrol_page\&utm_medium=banner\&utm_campaign=service)
## More about Patrol [#more-about-patrol]
* [How Patrol 4.0 Makes Cross-Platform Flutter Testing Possible][article_4x]
* [Simplifying Flutter Web Testing: Patrol Web][article_web]
* [Patrol VS Code Extension - A Better Way to Run and Debug Flutter UI Tests][article_vscode]
* GitHub Repository: [leancodepl/patrol][github_repo]
* Discord Channel: [Join the Patrol channel][dc_invite]
* Get a quick introduction to Patrol and see the video:
[leancode]: https://leancode.co
[native]: /documentation/native/overview
[finders]: /documentation/finders/overview
[hot restart]: /cli-commands/develop
[patrol_on_pubdev]: https://pub.dev/packages/patrol
[dc_invite]: https://discord.com/invite/ukBK5t4EZg
[article_web]: https://leancode.co/blog/patrol-web-support?utm_source=github.com&utm_medium=referral&utm_campaign=patrol_page
[article_4x]: https://leancode.co/blog/patrol-4-0-release?utm_source=github.com&utm_medium=referral&utm_campaign=patrol_page
[article_vscode]: https://leancode.co/blog/patrol-vs-code-extension?utm_source=github.com&utm_medium=referral&utm_campaign=patrol_page
[github_repo]: https://github.com/leancodepl/patrol
# Improved logging and reporting is here!
We’ve made some major improvements to how you can monitor and analyze your tests! With Patrol 3.13.0 and later, you’ll get:
* Verbose logging: Test names are now displayed in real time as they’re executed!
* Detailed step reporting: See every action Patrol takes during your test execution, giving you deeper insights into the process.
* Flutter logs in console: Now you can access Flutter logs directly within the patrol test output, streamlining debugging and analysis.
These enhancements will make it easier than ever to understand what's happening behind the code.
For a full breakdown of these updates, check out the [Logs and test results][logs] page!
[logs]: /documentation/logs
# Migrate from native to platform
Patrol 4.0 introduces the **Platform Automation API** and starts the deprecation of the legacy native automation entry points:
* `$.native` (Native Automation 1.0)
* `$.native2` (Native Automation 2.0)
`$.native`, `$.native2`, `NativeAutomator`, `NativeAutomator2`, and `NativeAutomatorConfig` are deprecated and will be removed in a future release. Migrate to `$.platform` and `PlatformAutomatorConfig`.
## Breaking changes you’ll likely hit [#breaking-changes-youll-likely-hit]
In Patrol 4.0 you should use `$.platform` as your entry point to platform automation.
* `$.platform.mobile`: cross-platform mobile actions (recommended default)
* `$.platform.android`: Android-only actions
* `$.platform.ios`: iOS-only actions
* `$.platform.web`: web automation
## Quick checklist [#quick-checklist]
* Upgrade the `patrol` package (and Patrol CLI) to 4.x.
* Rename your `integration_test/` directory to `patrol_test/` and update `.gitignore` for `test_bundle.dart` accordingly.
* If you want to keep using the old directory, you can configure it by adding `test_directory: integration_test` to your `patrol` section in `pubspec.yaml`.
Example:
```yaml title="pubspec.yaml"
patrol:
test_directory: integration_test
```
* Rewrite `$.native.*` and `$.native2.*` calls to `$.platform.*`.
* Replace `NativeSelector(...)` with `MobileSelector(...)`, `Selector(...)`, or `PlatformSelector(...)`.
* `patrol test` and `patrol develop` now prompt for a device. In CI, pass `--device`/`-d` to avoid the prompt, or set `CI=true` env variable.
## Common rewrites (search and replace) [#common-rewrites-search-and-replace]
### `native2` → `platform.mobile` [#native2--platformmobile]
#### Pressing Home [#pressing-home]
```dart
// Before:
await $.native2.pressHome(); // Press home for notifications.
// After:
await $.platform.mobile.pressHome(); // Press home for notifications.
```
#### Selector types you can pass to `$.platform.mobile.tap()` [#selector-types-you-can-pass-to-platformmobiletap]
Patrol 4.0 native actions (for example `$.platform.mobile.tap(...)`) accept a `MobileSelector`, `PlatformSelector`, or `Selector`. In practice, you’ll usually use one of these three types:
#### `MobileSelector(...)` (most common during migration) [#mobileselector-most-common-during-migration]
Use when Android and iOS require different selectors (this is the direct replacement for most `native2` uses).
```dart
// Before:
await $.native2.tap(
NativeSelector(
android: AndroidSelector(resourceName: 'com.example:id/submit_button'),
ios: IOSSelector(identifier: 'submitButton'),
),
);
// After:
await $.platform.mobile.tap(
MobileSelector(
android: AndroidSelector(resourceName: 'com.example:id/submit_button'),
ios: IOSSelector(identifier: 'submitButton'),
),
);
```
#### `Selector(...)` [#selector]
Use when the same selector “shape” works on Android and iOS (for example text-based selectors).
```dart
// Before:
await $.native.tap(Selector(text: 'Allow'));
// After:
await $.platform.mobile.tap(Selector(text: 'Allow'));
```
#### `PlatformSelector(...)` [#platformselector]
Use when you want to provide selectors for Android + iOS + Web in one place. This is most useful with `$.platform.tap(...)` (platform router), but it can also be used with `$.platform.mobile.*` as long as it includes Android and iOS selectors.
```dart
// Before:
await $.native2.tap(
NativeSelector(
android: AndroidSelector(text: 'Click Android text'),
ios: IOSSelector(text: 'Click iOS text'),
// Web was not supported in native2
),
);
// After:
await $.platform.tap(
PlatformSelector(
android: AndroidSelector(text: 'Click Android text'),
ios: IOSSelector(text: 'Click iOS text'),
web: WebSelector(text: 'Click web text'),
),
);
```
#### Platform-specific actions [#platform-specific-actions]
Some legacy calls were platform-specific but lived under `$.native`. In Patrol 4.0 they live under `$.platform.android` / `$.platform.ios`.
```dart
// Before:
await $.native.pressBack();
// After:
await $.platform.android.pressBack();
```
```dart
// Before:
await $.native.closeHeadsUpNotification();
// After:
await $.platform.ios.closeHeadsUpNotification();
```
## More examples [#more-examples]
### NativeAutomator → PlatformAutomator [#nativeautomator--platformautomator]
If you used to create native automators manually, create a `PlatformAutomator` instead.
```dart
// Before:
patrolTearDown(() async {
final automator = NativeAutomator(config: nativeConfig);
await automator.enableWifi();
await automator.enableCellular();
});
// After:
patrolTearDown(() async {
final automator = PlatformAutomator(config: platformConfig);
await automator.mobile.enableWifi();
await automator.mobile.enableCellular();
});
```
### NativeAutomatorConfig → PlatformAutomatorConfig [#nativeautomatorconfig--platformautomatorconfig]
```dart
// Before:
final nativeConfig = NativeAutomatorConfig(
keyboardBehavior: KeyboardBehavior.alternative,
);
// After:
final platformConfig = PlatformAutomatorConfig.fromOptions(
keyboardBehavior: KeyboardBehavior.alternative,
);
```
# New package - patrol_finders
We're introducing [`patrol_finders`] - a new package in Patrol framework! It was
created to make it easier to use Patrol finders in widget tests.
We decided to separate out our finders mechanism to another package, so
developers who would like to use Patrol's awesome finders in their widget tests
don't need to depend on whole Patrol package. This way you can conveniently use
Patrol finders in widget or golden tests, whichever platforms you need to
support in your project!
### How to use it? [#how-to-use-it]
We made a short tutorial on how to use `patrol_finders` package separately in
widget tests, you can find it in [Using Patrol finders in widget tests] section.
### Does this change affect my Patrol tests? [#does-this-change-affect-my-patrol-tests]
If you have already some Patrol tests in your project, there are no breaking
changes in this release - everything works the same as before. Though you may
see some deprecation warnings in your code - you can get your code aligned with
them to prepare for future changes.
[Using Patrol finders in widget tests]: /documentation/finders/finders-setup
[`patrol_finders`]: https://pub.dev/packages/patrol_finders
# Patrol MCP is here!
Patrol MCP is an MCP (Model Context Protocol) server that lets AI assistants run and manage Patrol tests in your Flutter projects. Check out our [release article on LeanCode's blog](https://leancode.co/blog/patrol-mcp-release).
With Patrol MCP, AI assistants like Claude, Cursor, Copilot, and Gemini can:
* Run and re-run individual Patrol tests
* Capture screenshots with auto-detected platform
* Read native UI tree during active sessions
We also added documentation to help you get started:
* [Patrol MCP documentation & setup guide](/documentation/other/patrol-mcp)
* [patrol\_mcp on pub.dev](https://pub.dev/packages/patrol_mcp)
If you have any questions or want to submit feedback, head on to [GitHub](https://github.com/leancodepl/patrol) or [our Discord server](https://discord.gg/ukBK5t4EZg).
# Patrol 3.0 is here
Patrol 3.0 is the new major version of Patrol.
## `patrol v3` and DevTools extension [#patrol-v3-and-devtools-extension]
The highlight of this release is the **Patrol DevTools Extension**. We created
it to enhance your UI test development experience with `patrol develop` by
making it much easier to explore the native view hierarchy. With Patrol's new
DevTools extension, you can effortlessly inspect the currently visible
Android/iOS views and discover their properties. This information can be then
used in native selectors like `$.native.tap()`, eliminating the need for
external tools.
Patrol is one of the first packages in the whole Flutter ecosystem to have a
DevTools extension. We have started working on it as soon as the Flutter team
has announced that they're working on making DevTools extensible. We immediately
realized how powerful this feature is and how it can enable us to deliver better
UI testing experience.
This is, of course, just the beginning, and we have plans to introduce more
features in future updates of our DevTools extension.
### Changes in `patrol` v3: [#changes-in-patrol-v3]
The DevTools extension is not the only new feature in this release. Other
changes include:
* **Minimum Flutter version**: The minimum supported Flutter version has been
bumped to 3.16 to make it compatible with a few breaking changes that were
introduced to the `flutter_test` package that `patrol` and `patrol_finders`
depend on. We hope you'll have an easy time upgrading to 3.16, but if not, you
can always use Patrol v2 until you're ready to upgrade.
* **A few breaking changes**:
* The `bindingType` parameter has been removed from the `patrolTest()`
function. Now, only `PatrolBinding` is used and it's also automatically
initialized.
* The `nativeAutomation` parameter has also been removed from the
`patrolTest()` function. Now `patrolTest()` implies native automation and
you can use `patrolWidgetTest()` if you don't need it.
* `PatrolTester` class has been renamed to `PatrolIntegrationTester`. Now
`PatrolTester` is used with `patrolWidgetTest()` *without* native automation
and `PatrolIntegrationTester` is used with `patrolTest()` *with* native
automation.
* **Patrol CLI version requirement**: Patrol v3 requires Patrol CLI v2.3.0 or
newer, so make sure to `patrol update`!
## `patrol_finders` v2 [#patrol_finders-v2]
Along with `patrol` v3, we are releasing the v2 of [patrol\_finders][patrol_finders]. In case you
missed it, we split `patrol_finders` from `patrol` a few months ago in response
to our community members who loved Patrol's lean finders syntax, but weren't
interested in developing integration tests. [Here's the docs
page][patrol_finders_docs] about `patrol_finders` in case you missed it.
### Changes in `patrol_finders` v2: [#changes-in-patrol_finders-v2]
* **Minimum Flutter version**: The minimum supported Flutter version of
`patrol_finders` has been bumped to 3.16, just like in `patrol`'s case.
* The deprecated `andSettle` method has been removed from all `PatrolTester` and
`PatrolFinder` methods like `tap()`, `enterText()`, and so on. Developers
should now use `settlePolicy` as a replacement, which has been available since
June.
* The default `settlePolicy` has been changed to [SettlePolicy.trySettle].
## Wrapping up [#wrapping-up]
As you can see, these updates have a little bit of everything - a large new
feature, support for the latest Flutter version, and a clean-up of a few
deprecations. We encourage you to explore our new DevTools extension and look
forward to your feedback and ideas for new features as we continue to evolve the
Patrol ecosystem. Meanwhile, we're getting back to work on Patrol, with a single
goal in mind – to make it the go-to UI testing framework for Flutter apps.
[patrol_finders]: https://pub.dev/packages/patrol_finders
[patrol_finders_docs]: https://patrol.leancode.co/patrol-finders-release
[SettlePolicy.trySettle]: https://pub.dev/documentation/patrol_finders/latest/patrol_finders/SettlePolicy.html#trySettle
# New major release - Patrol 4.0
Patrol 4.0 is live! Check out our [release article on LeanCode's blog](https://leancode.co/blog/patrol-4-0-release).
Read more about new features coming in this release in deep-dives articles:
* [Simplifying Flutter Web Testing: Patrol Web](https://leancode.co/blog/patrol-web-support)
* [Patrol VS Code Extension - A Better Way to Run and Debug Flutter UI Tests](https://leancode.co/blog/patrol-vs-code-extension)
We also updated documentation if you're looking for guides and examples:
* [Native to Platform Migration guide (4.0)](/native-to-platform-migration)
* [see how to test on Web with Patrol](/documentation/web)
* [find out what features offers our VS Code extension](/documentation/other/patrol-devtools-extension)
* [learn how to use `platform` instead of `native`](/documentation/native/usage)
Run `patrol update` and upgrade `patrol` in your pubspec to use the newest version of Patrol.
You can find the VS Code extension in Marketplace inside VS Code.
We're hoping this early Christmas present will make your mobile testing easier!
If you have any questions or want to submit feedback, head on to [GitHub](https://github.com/leancodepl/patrol) or [our Discord server](https://discord.gg/ukBK5t4EZg). See you there!
# Articles & Resources
Patrol, created by LeanCode, is here to make UI testing for Flutter apps faster and more reliable. What started as a simple tool has grown into a widely used testing framework supported by an active community. To help you get the most out of Patrol, we've gathered some valuable resources—articles and insights—crafted by both our team and fellow community members. Check them out below!
If you wrote an article featuring Patrol, let us know on our [Discord channel in #articles](https://discord.com/channels/1167030497612922931/1341768486409605211). This way, we will be able to grow this list.
### Official Patrol resources [#official-patrol-resources]
* [Flutter UI Testing with Patrol](https://resocoder.com/2024/12/02/flutter-ui-testing-with-patrol/) by Reso Coder
* [How to Test Native Features in Flutter Apps with Patrol and Codemagic](https://blog.codemagic.io/how-to-test-native-features-in-flutter-apps-with-patrol-and-codemagic/) by Codemagic
* [Observable Flutter #56 - Testing with Patrol](https://www.youtube.com/live/fidNg4ZzUKA?si=QyzklPI8wUQgyf0V) by Flutter.dev
* [What's Patrol and why you should use it](https://youtu.be/KRWgAonXH9o?si=vHiqGZfnP5q8GFBL) by Mateusz Wojtczak (LeanCode) at Full Stack Flutter 2024
* [What's Patrol and why you should use it: tests without a hassle](https://youtu.be/v5j01RKAseM?si=yrlK0x4YGpszHPJE) by Mateusz Wojtczak (LeanCode) at Flutter Heroes 2024
* [Give Patrol a Try: Hands-On in 10 Minutes or Less](https://leancode.co/blog/try-patrol-quick-hands-on-tutorial) by LeanCode
* [Patrol MCP Release](https://leancode.co/blog/patrol-mcp-release) by LeanCode
### Community-created articles [#community-created-articles]
* [Flutter End-to-End Test Using Patrol (AlloFresh Case Study)](https://medium.com/allofresh-engineering/flutter-end-to-end-test-using-patrol-810c6a25bf8d)
* [How to make Allure work with Patrol tests in CI/CD](https://medium.com/@kolbevich/how-to-make-allure-work-with-patrol-tests-in-ci-cd-a03800fbe223)
* [How to write your Flutter Integration tests using Patrol](https://dev.to/codedigga/how-to-write-your-flutter-integration-tests-using-patrol-4l4d)
* [When Harry Liked Sally: speeding up a multi-user UI test with Firebase auth API](https://medium.com/@kolbevich/when-harry-likes-sally-speeding-up-a-multi-user-ui-test-with-firebase-auth-api-7f1edd724e81)
* [Patrol‑Driven UI Test Architecture for Flutter](https://vbacik-10.medium.com/patrol-driven-ui-test-architecture-for-flutter-2e92923cfa49)
* [Patrol Integration Testing: Accelerating Flutter App Development with Confidence](https://www.willowtreeapps.com/craft/patrol-integration-testing-accelerating-flutter-app-development-with-confidence)
### Patrol vs. other testing frameworks [#patrol-vs-other-testing-frameworks]
* [Flutter Testing: Where to Start Without Losing Your Mind](https://medium.com/@samin.sheyda4/flutter-testing-where-to-start-without-losing-your-mind-dfee30755ea9)
# build
### Synopsis [#synopsis]
Build app binaries for integration testing.
```
patrol build android
patrol build ios
```
To see all available options and flags, run `patrol build android --help` or
`patrol build ios --help`.
For `patrol build` to work, you must complete [native setup].
### Description [#description]
`patrol build` is useful if you want to run test on CI, for example on Firebase
Test Lab. It works the same as `patrol test`, except that it does not run tests.
`patrol build` builds apps in debug mode by default.
To run tests on a physical iOS device on a device farm, the apps have to be
built in release mode. To do so, pass the `--release` flag.
### Examples [#examples]
**To build a single test for Android in debug mode**
```
patrol build android --target patrol_test/example_test.dart
```
or alternatively (but redundantly):
```
patrol build android --target patrol_test/example_test.dart --debug
```
**To build all tests for Android in debug mode**
```
patrol build android
```
**To build a single test for iOS device in release mode**
```
patrol build ios --target patrol_test/example_test.dart --release
```
**To build a single test for iOS simulator in debug mode**
```
patrol build ios --target patrol_test/example_test.dart --debug
```
**To build with custom build name and number**
```
patrol build android --build-name=1.2.3 --build-number=123
patrol build ios --build-name=1.2.3 --build-number=123 --release
```
**To build with full isolation between tests**
```
patrol build ios --full-isolation
```
The `--full-isolation` flag enables full isolation between test runs on iOS Simulator.
### Under the hood [#under-the-hood]
The `patrol build` command walks through hierarchy of the `patrol_test`
directory and finds all files that end with `_test.dart`, and then creates an
additional "test bundle" file that references all the tests it found. Thanks to
this, all tests are built into a single app binary - only a single build is
required, which greatly reduces time spent on building. Then, it runs a new app
process for every test, improving isolation between tests and enabling sharding.
We call this feature **advanced test bundling**. It provides deep and seamless
integration with existing Android and iOS testing tools. It also fixes some
long-standing Flutter issues:
* [#115751](https://github.com/flutter/flutter/issues/115751)
* [#101296](https://github.com/flutter/flutter/issues/101296)
* [#117386](https://github.com/flutter/flutter/issues/117386)
We think that **this is huge** (even though it may not look like it at first
glance). To learn more, read [the in-depth technical article][patrol_v2_article]
explaining the nuts and bolts.
[patrol_v2_article]: https://leancode.co/blog/patrol-2-0-improved-flutter-ui-testing
[native setup]: /documentation
# devices
### Synopsis [#synopsis]
List attached devices, simulators and emulators
```
patrol devices
```
To see all available options and flags, run `patrol devices --help`.
### Description [#description]
It's intended to be a simpler, Patrol-aware alternative to `flutter devices`.
# doctor
### Synopsis [#synopsis]
Show information about installed tooling.
```
patrol doctor
```
To see all available options and flags, run `patrol doctor --help`.
# develop
### Synopsis [#synopsis]
Develop integration tests with Hot Restart.
```
patrol develop
```
To see all available options and flags, run `patrol develop --help`.
### Description [#description]
`patrol develop` makes the development of integration tests faster and more fun,
thanks to Flutter's Hot Restart feature.
To run a test file with Hot Restart, specify the `--target` option:
```
patrol develop --target patrol_test/example_test.dart
```
This performs a build of your app, so it's usually slow.
When the build completes and the app starts, Hot Restart becomes active after a
short while. Once it is active, type **R** to trigger a Hot Restart.
You can specify custom build number and build name using the `--build-name` and `--build-number` flags.
```
patrol develop --target patrol_test/example_test.dart --build-name=1.2.3 --build-number=123
```
### Demo [#demo]
### Caveats [#caveats]
`patrol develop` is powerful, but it has some limitations. It's important to
understand them to write correct tests.
Flutter apps consist of 2 parts: the native part and the Flutter part. **Hot
Restart restarts only the Flutter part of your app**.
When you press **R**, the Flutter part of your app is restarted – your `main()`
function is run again.
It's important to note that:
* The native part of your app is not restarted
* The app's data is not cleared
* The app is not uninstalled
Below are a few common scenarios when state of your app will probably differ
between the first and later Hot Restarts.
**No support for physical iOS**
Patrol's Hot Restart is very unreliable when running on physical iOS devices, to
the point that we consider it completely broken. This is unfortunate, but it's a
[bug on the Flutter's side](https://github.com/flutter/flutter/issues/122698).
**Permissions**
Once granted, a permissions cannot be revoked.
You have to work around this by handling both cases (of granted and not granted
permissions) in your tests.
Removing all permissions at the beginning of each test won't work - both iOS and
Android kill the app when any permission is revoked, and when the app dies, Hot
Restart stops.
**File system**
The files your app creates in its internal storage aren't cleared between hot
restarts.
For example, when some data is saved to SharedPreferences during the first
restart, it will stay around in subsequent restarts (unless it's manually
cleared).
The same goes for any data the app creates during test - photos, documents, etc.
It's your responsibility to clean them up at the right time to have a stable
environment during `patrol develop`.
**Native state**
If your app has some native code that runs only when the app is first run, it
won't be re-executed on hot restarts.
These cases are quite specific and it's hard to give advice without knowing the
context.
See also:
* [The difference between Hot Restart and Hot Reload in Flutter][so_question]
[so_question]: https://stackoverflow.com/q/61787776/7009800
# test
### Synopsis [#synopsis]
Run integration tests.
```
patrol test
```
To see all available options and flags, run `patrol test --help`.
### Description [#description]
This command is the one use you'll be using most often.
`patrol test` does the following things:
1. Builds the app under test (AUT) and the instrumentation app
2. Installs the AUT and the instrumentation on the selected device
3. Runs the tests natively, and reports results back in native format.
Under the hood, it calls Gradle (when testing on Android) and `xcodebuild` (when
testing on iOS).
### Discussion [#discussion]
By default, `patrol test` runs all integration tests (files ending with
`_test.dart` located in the `patrol_test` directory). You can customize the test directory by setting `test_directory` in your `pubspec.yaml` under the `patrol` section.
To run a single test, use `--target`:
```
patrol test --target patrol_test/login_test.dart
```
You can use `--target` more than once to run multiple tests:
```
patrol test \
--target patrol_test/login_test.dart \
--target patrol_test/app_test.dart
```
Or alternatively:
```
patrol test --targets patrol_test/login_test.dart,patrol_test/app_test.dart
```
Test files must end with `_test.dart`. Otherwise the file is not considered a
test and is not run.
There's no difference between
`--target`
and
`--targets`
.
### Tags [#tags]
You can use tags to run only tests with specific tags.
First specify tags in your patrol tests:
```dart
patrol(
'example test with tag',
tags: ['android'],
($) async {
await createApp($);
await $(FloatingActionButton).tap();
expect($(#counterText).text, '1');
},
);
patrol(
'example test with two tags',
tags: ['android', 'ios'],
($) async {
await createApp($);
await $(FloatingActionButton).tap();
expect($(#counterText).text, '1');
},
);
```
Then you can run tests with the tags you specified:
```bash
patrol test --tags android
patrol test --tags=android
patrol test --tags='android||ios'
patrol test --tags='(android || ios)'
patrol test --tags='(android && tablet)'
```
You can also use `--exclude-tags` to exclude tests with specific tags:
```bash
patrol test --exclude-tags android
patrol test --exclude-tags='(android||ios)'
```
For comprehensive information about tag syntax, complex expressions, and advanced usage, see
the [Patrol tags documentation](https://patrol.leancode.co/documentation/patrol-tags).
### Coverage [#coverage]
Coverage collection is currently not supported on macOS.
To collect coverage from patrol tests, use `--coverage`.
```
patrol test --coverage
```
The LCOV report will be saved to `/coverage/patrol_lcov.info`.
Additionally, you can exclude certain files from the report using glob patterns and `--coverage-ignore` option.
For instance,
```
patrol test --coverage --coverage-ignore="**/*.g.dart"
```
excludes all files ending with `.g.dart`.
### Build versioning [#build-versioning]
You can specify custom build number and build name using the `--build-name` and `--build-number` flags. These work
the same as in Flutter CLI:
* `--build-name`: Version name of the app. (e.g. `1.2.3`)
* `--build-number`: Version code of the app. (e.g. `123`)
```
patrol test --build-name=1.2.3 --build-number=123
patrol test --target patrol_test/login_test.dart --build-name=1.2.3 --build-number=123
```
### Isolation of test runs [#isolation-of-test-runs]
To achieve full isolation between test runs:
* On Android: set `clearPackageData` to `true` in your `build.gradle` file,
* On iOS Simulator: use the `--full-isolation` flag
This functionality is experimental on iOS and might be removed in the future releases.
```bash
patrol test --full-isolation
```
### Web Platform [#web-platform]
Patrol supports running tests on Flutter web using Playwright. To run tests on web:
```bash
patrol test --device chrome
```
When running on web:
* Tests execute in Chromium browser via Playwright
* CLI arguments can be used to configure Playwright
* Test results are generated in `test-results/`
#### Arguments [#arguments]
Playwright configuration is updated with values passed to the command. This allows direct control over Playwright
features such as reporting. To see the full list of supported arguments, run `patrol test --help`.
**Note:** Some arguments are not supported on web:
* `--flavor`: Flavors are not supported for Flutter web
* `--uninstall`: Not applicable to web platform
* `--clear-permissions`: Not applicable to web platform
* `--full-isolation`: Not applicable to web platform
### Under the hood [#under-the-hood]
`patrol test` basically calls `patrol build` and then runs the built app
binaries. For more info, read [docs of `patrol build`][patrol_build].
[patrol_build]: /cli-commands/build
# update
### Synopsis [#synopsis]
Update Patrol CLI to the latest version.
```
patrol update
```
To see all available options and flags, run `patrol update --help`.
# Get in Touch with the Patrol Team
We're here to help you get the most out of Patrol. Whether you've run into an issue, have a question, or just want to share your thoughts, you're in the right place. Choose the option that fits your needs best:
* **Found a bug?** [Create an issue on GitHub](https://github.com/leancodepl/patrol/issues/new) so we can review it.
* **Not sure how to use something, or it's not working?** [Join our Discord community](https://discord.gg/ukBK5t4EZg) and ask your question in the dedicated channel.
* **Want to share feedback?** Let us know what you think through [our feedback survey](https://form.typeform.com/to/iUpJOKj6).
* **Need training, a professional setup, or a Patrol consultant?** [Contact us directly](https://leancode.co/get-estimate?utm_source=patrol_page\&utm_medium=contact) to discuss how we can support your team.
* **Want to find out more about our Automated UI testing with Patrol service?** [Visit our service page](https://leancode.co/products/automated-ui-testing-in-flutter?utm_source=patrol_page\&utm_medium=contact) dedicated to automating QA processes in your Flutter app.
**Important Note:** Patrol is an open-source project maintained by LeanCode alongside our commercial work, so while we aim to respond and assist as much as possible, our time can be limited. We hope the above-mentioned information will help you find the right channel for your needs.
# Cheat sheet
# Compatibility table
The following table describes which versions of `patrol`
and `patrol_cli` are compatible with each other.
The simplest way to ensure that both packages are compatible
is by always using the latest version. However,
if for some reason that isn't possible, you can refer to
the table below to assess which version you should use.
This table shows the compatible versions between patrol\_cli and patrol packages.
| patrol\_cli version | patrol version | Minimum Flutter version |
| ------------------- | --------------- | ----------------------- |
| 4.4.0+ | 4.6.0+ | 3.32.0 |
| 4.3.0 - 4.3.1 | 4.5.0 | 3.32.0 |
| 4.2.0 | 4.2.0 - 4.4.0 | 3.32.0 |
| 4.0.2 - 4.1.0 | 4.1.0 - 4.1.1 | 3.32.0 |
| 4.0.0 - 4.0.1 | 4.0.0 - 4.0.1 | 3.32.0 |
| 3.11.0 | 3.20.0 | 3.32.0 |
| 3.9.0 - 3.10.0 | 3.18.0 - 3.19.0 | 3.32.0 |
| 3.7.0 - 3.8.0 | 3.16.0 - 3.17.0 | 3.32.0 |
| 3.5.0 - 3.6.0 | 3.14.0 - 3.15.2 | 3.24.0 |
| 3.4.1 | 3.13.1 - 3.13.2 | 3.24.0 |
| 3.4.0 | 3.13.0 | 3.24.0 |
| 3.3.0 | 3.12.0 | 3.24.0 |
| 3.2.1 | 3.11.2 | 3.24.0 |
| 3.2.0 | 3.11.0 - 3.11.1 | 3.22.0 |
| 3.1.0 - 3.1.1 | 3.10.0 | 3.22.0 |
| 2.6.5 - 3.0.1 | 3.6.0 - 3.10.0 | 3.16.0 |
| 2.6.0 - 2.6.4 | 3.4.0 - 3.5.2 | 3.16.0 |
| 2.3.0 - 2.5.0 | 3.0.0 - 3.3.0 | 3.16.0 |
| 2.2.0 - 2.2.2 | 2.3.0 - 2.3.2 | 3.3.0 |
| 2.0.1 - 2.1.5 | 2.0.1 - 2.2.5 | 3.3.0 |
| 2.0.0 | 2.0.0 | 3.3.0 |
| 1.1.4 - 1.1.11 | 1.0.9 - 1.1.11 | 3.3.0 |
## Notes [#notes]
* Versions marked with `+` indicate compatibility with all later versions
* Ranges (e.g., `2.0.0 - 2.1.0`) indicate compatibility with all versions in that range
* The minimum Flutter version is required for both packages to work correctly
# Install Patrol
Check out our video version of this tutorial on YouTube!
If you want to use Patrol finders in your existing widget or golden
tests, go to [Using Patrol finders in widget tests].
## Setup [#setup]
Install `patrol_cli`:
```
flutter pub global activate patrol_cli
```
[Patrol CLI] (command-line interface) is a small program that enables running
Patrol UI tests. It is necessary to run UI tests (`flutter test` won't work! [Here's why]).
Make sure to add `patrol` to your `PATH` environment variable.
It's explained in the [README].
Verify that installation was successful and your environment is set up properly:
```
patrol doctor
```
Example output:
```
Patrol CLI version: 2.3.1+1
Android:
• Program adb found in /Users/username/Library/Android/sdk/platform-tools/adb
• Env var $ANDROID_HOME set to /Users/username/Library/Android/sdk
iOS / macOS:
• Program xcodebuild found in /usr/bin/xcodebuild
• Program ideviceinstaller found in /opt/homebrew/bin/ideviceinstaller
Web:
• Program node found in /usr/bin/node
• Program npm found in /usr/bin/npm
```
Be sure that for the platform you want to run the test on, all the checks are green.
Patrol CLI invokes the Flutter CLI for certain commands. To override the command used,
pass the `--flutter-command` argument or set the `PATROL_FLUTTER_COMMAND` environment
variable. This supports FVM (by setting the value to `fvm flutter`), puro (`puro flutter`)
and potentially other version managers.
Add a dependency on the `patrol` package in the
`dev_dependencies` section of `pubspec.yaml`. `patrol` package requires
Android SDK version 21 or higher.
```
flutter pub add patrol --dev
```
Create `patrol` section in your `pubspec.yaml`:
```yaml title="pubspec.yaml"
dependencies:
# ...
dev_dependencies:
# ...
patrol:
app_name: My App
android:
package_name: com.example.myapp
ios:
bundle_id: com.example.MyApp
macos:
bundle_id: com.example.macos.MyApp
```
**Test Directory Configuration**: By default, Patrol looks for tests in the `patrol_test/` directory.
You can customize this by adding `test_directory: your_custom_directory` to your `patrol` section in `pubspec.yaml`.
```yaml title="pubspec.yaml (optional customization)"
patrol:
app_name: My App
test_directory: integration_test # Use custom directory
android:
package_name: com.example.myapp
ios:
bundle_id: com.example.MyApp
macos:
bundle_id: com.example.macos.MyApp
```
In this tutorial, we are using example app, which has package name
`com.example.myapp` on Android, bundle id `com.example.MyApp` on iOS,
`com.example.macos.MyApp` on macOS and `My App` name on all platforms.
Replace any occurences of those names with proper values.
If you don't know where to get `package_name` and `bundle_id` from, see the [FAQ] section.
Integrate with native side
The 3 first steps were common across platforms. The rest is platform-specific.
Psst... Android is a bit easier to set up, so we recommend starting with it!
Go to **android/app/src/androidTest/java/com/example/myapp/** in your project
directory. If there are no such folders, create them. **Remember to replace
`/com/example/myapp/` with the path created by your app's package name.**
Create a file named `MainActivityTest.java` and copy there the code below.
```java title="MainActivityTest.java"
package com.example.myapp; // replace "com.example.myapp" with your app's package
import androidx.test.platform.app.InstrumentationRegistry;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.junit.runners.Parameterized;
import org.junit.runners.Parameterized.Parameters;
import pl.leancode.patrol.PatrolJUnitRunner;
@RunWith(Parameterized.class)
public class MainActivityTest {
@Parameters(name = "{0}")
public static Object[] testCases() {
PatrolJUnitRunner instrumentation = (PatrolJUnitRunner) InstrumentationRegistry.getInstrumentation();
// replace "MainActivity.class" with "io.flutter.embedding.android.FlutterActivity.class"
// if in AndroidManifest.xml in manifest/application/activity you have
// android:name="io.flutter.embedding.android.FlutterActivity"
instrumentation.setUp(MainActivity.class);
instrumentation.waitForPatrolAppService();
return instrumentation.listDartTests();
}
public MainActivityTest(String dartTestName) {
this.dartTestName = dartTestName;
}
private final String dartTestName;
@Test
public void runDartTest() {
PatrolJUnitRunner instrumentation = (PatrolJUnitRunner) InstrumentationRegistry.getInstrumentation();
instrumentation.runDartTest(dartTestName);
}
}
```
Go to the **build.gradle.kts** file, located in **android/app** folder in your
project directory.
Add these 2 lines to the `defaultConfig` section:
```kotlin title="android/app/build.gradle.kts"
testInstrumentationRunner = "pl.leancode.patrol.PatrolJUnitRunner"
testInstrumentationRunnerArguments["clearPackageData"] = "true"
```
Add this section to the `android` section:
```kotlin title="android/app/build.gradle.kts"
testOptions {
execution = "ANDROIDX_TEST_ORCHESTRATOR"
}
```
Add this line to `dependencies` section:
```kotlin title="android/app/build.gradle.kts"
androidTestUtil("androidx.test:orchestrator:1.5.1")
```
Bear in mind that ProGuard can lead to some problems if not well configured, potentially causing issues such as `ClassNotFoundException`s.
Keep all the Patrol packages or disable ProGuard in `android/app/build.gradle.kts`:
```kotlin title="android/app/build.gradle.kts"
...
buildTypes {
getByName("release") {
...
}
getByName("debug") {
isMinifyEnabled = false
isShrinkResources = false
}
}
```
Go to **android/app/src/androidTest/java/com/example/myapp/** in your project
directory. If there are no such folders, create them. **Remember to replace
`/com/example/myapp/` with the path created by your app's package name.**
Create a file named `MainActivityTest.java` and copy there the code below.
```java title="MainActivityTest.java"
package com.example.myapp; // replace "com.example.myapp" with your app's package
import androidx.test.platform.app.InstrumentationRegistry;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.junit.runners.Parameterized;
import org.junit.runners.Parameterized.Parameters;
import pl.leancode.patrol.PatrolJUnitRunner;
@RunWith(Parameterized.class)
public class MainActivityTest {
@Parameters(name = "{0}")
public static Object[] testCases() {
PatrolJUnitRunner instrumentation = (PatrolJUnitRunner) InstrumentationRegistry.getInstrumentation();
// replace "MainActivity.class" with "io.flutter.embedding.android.FlutterActivity.class"
// if in AndroidManifest.xml in manifest/application/activity you have
// android:name="io.flutter.embedding.android.FlutterActivity"
instrumentation.setUp(MainActivity.class);
instrumentation.waitForPatrolAppService();
return instrumentation.listDartTests();
}
public MainActivityTest(String dartTestName) {
this.dartTestName = dartTestName;
}
private final String dartTestName;
@Test
public void runDartTest() {
PatrolJUnitRunner instrumentation = (PatrolJUnitRunner) InstrumentationRegistry.getInstrumentation();
instrumentation.runDartTest(dartTestName);
}
}
```
Go to the **build.gradle** file, located in **android/app** folder in your
project directory.
Add these 2 lines to the `defaultConfig` section:
```groovy title="android/app/build.gradle"
testInstrumentationRunner "pl.leancode.patrol.PatrolJUnitRunner"
testInstrumentationRunnerArguments clearPackageData: "true"
```
Add this section to the `android` section:
```groovy title="android/app/build.gradle"
testOptions {
execution "ANDROIDX_TEST_ORCHESTRATOR"
}
```
Add this line to `dependencies` section:
```groovy title="android/app/build.gradle"
androidTestUtil "androidx.test:orchestrator:1.5.1"
```
Bear in mind that ProGuard can lead to some problems if not well configured, potentially causing issues such as `ClassNotFoundException`s.
Keep all the Patrol packages or disable ProGuard in `android/app/build.gradle`:
```groovy title="android/app/build.gradle"
...
buildTypes {
release {
...
}
debug {
minifyEnabled false
shrinkResources false
}
}
```
Open `ios/Runner.xcworkspace` in Xcode.
Create a test target if you do not already have one (see the screenshot below
for the reference). Select `File > New > Target...` and select `UI Testing Bundle`.
Change the `Product Name` to `RunnerUITests`. Set the `Organization Identifier`
to be the same as for the `Runner` (no matter if you app has flavors or not).
For our example app, it's `com.example.MyApp` just as in the `pubspec.yaml` file.
Make sure `Target to be Tested` is set to `Runner` and language is set to `Objective-C`.
Select `Finish`.
2 files are created: `RunnerUITests.m` and `RunnerUITestsLaunchTests.m`.
Delete `RunnerUITestsLaunchTests.m` **through Xcode** by clicking on it and
selecting `Move to Trash`.
Make sure that the **iOS Deployment Target** of `RunnerUITests` within the
**Build Settings** section is the same as `Runner`.
The minimum supported **iOS Deployment Target** is `13.0`.
Replace contents of `RunnerUITests.m` file with the following:
```objective-c title="ios/RunnerUITests/RunnerUITests.m"
@import XCTest;
@import patrol;
@import ObjectiveC.runtime;
#if !defined(PATROL_INTEGRATION_TEST_IOS_RUNNER)
#import "PatrolIntegrationTestIosRunner.h"
#endif
PATROL_INTEGRATION_TEST_IOS_RUNNER(RunnerUITests)
```
Add the newly created target to `ios/Podfile` by embedding in the existing
`Runner` target.
```ruby title="ios/Podfile"
target 'Runner' do
# Do not change existing lines.
...
target 'RunnerUITests' do
inherit! :complete
end
end
```
If you haven't yet migrated your project to Swift Package Manager, follow the Flutter docs for
[integrating SPM with your app](https://docs.flutter.dev/packages-and-plugins/swift-package-manager/for-app-developers).
Following the Flutter docs will allow you to run or build the app itself (Runner target) but we still need to do some configuration for Patrol tests (RunnerUITests target).
To add the `FlutterGeneratedPluginSwiftPackage` to the **RunnerUITests**
target, in Xcode go to **RunnerUITests > General > Frameworks and Libraries**,
click **+**, and select `FlutterGeneratedPluginSwiftPackage`.
Create an empty file `patrol_test/example_test.dart` in the root of your Flutter project. From the command line, run
the following command and make sure it completes with no errors:
```
flutter build ios --config-only patrol_test/example_test.dart
```
**(CocoaPods only)** Go to your `ios` directory and run:
```
pod install --repo-update
```
Open your Xcode project and Make sure that for each build configuration,
the `RunnerUITests` have the same Configuration Set selected as the `Runner`:
Go to **RunnerUITests** -> **Build Phases** and add 2 new "Run Script Phase" Build Phases.
Name them `xcode_backend build` and `xcode_backend embed_and_thin`.
Arrange the newly created Build Phases in the order shown in the screenshot below.
Paste this code into the `xcode_backend build` Build Phase:
```
/bin/sh "$FLUTTER_ROOT/packages/flutter_tools/bin/xcode_backend.sh" build
```
Paste this code into the `xcode_backend embed_and_thin` Build Phase:
```
/bin/sh "$FLUTTER_ROOT/packages/flutter_tools/bin/xcode_backend.sh" embed_and_thin
```
Xcode by default also enables a "parallel execution" setting, which
breaks Patrol. Disable it **for all schemes** (if you have more than one):
Go to **RunnerUITests** -> **Build Settings**, search for **User Script Sandboxing**
and make sure it's set to **No**.
You're ready to run tests on iOS simulator but using real devices is a bit
more complicated.
Check out [Setup for physical iOS Devices] to learn how to set up Patrol for physical iOS devices.
Support for macOS is in alpha stage. Please be aware that some features
may not work as expected. There is also no native automation support
for macOS yet. If you encounter any issues, please report them on
GitHub.
Open `macos/Runner.xcworkspace` in Xcode.
Create a test target if you do not already have one via `File > New > Target...`
and select `UI Testing Bundle`. Change the `Product Name` to `RunnerUITests`. Make
sure `Target to be Tested` is set to `Runner` and language is set to `Objective-C`.
Select `Finish`.
2 files are created: `RunnerUITests.m` and `RunnerUITestsLaunchTests.m`.
Delete `RunnerUITestsLaunchTests.m` **through Xcode**.
Make sure that the **macOS Deployment Target** of `RunnerUITests` within the
**Build Settings** section is the same as `Runner`.
The minimum supported **macOS Deployment Target** is `10.14`.
Replace contents of `RunnerUITests.m` file with the following:
```objective-c title="macos/RunnerUITests/RunnerUITests.m"
@import XCTest;
@import patrol;
@import ObjectiveC.runtime;
PATROL_INTEGRATION_TEST_MACOS_RUNNER(RunnerUITests)
```
Add the newly created target to `macos/Podfile` by embedding in the existing
`Runner` target.
```ruby title="macos/Podfile"
target 'Runner' do
# Do not change existing lines.
...
target 'RunnerUITests' do
inherit! :complete
end
end
```
Create an empty file `patrol_test/example_test.dart` in the root of your Flutter project. From the command line, run:
```
flutter build macos --config-only patrol_test/example_test.dart
```
Go to your `macos` directory and run:
```
pod install --repo-update
```
Go to **RunnerUITests** -> **Build Phases** and add 2 new "Run Script Phase" Build Phases.
Rename them to `xcode_backend build` and `xcode_backend embed_and_thin` by double clicking
on their names.
Arrange the newly created Build Phases in the order shown in the screenshot below.
Paste this code into the first `macos_assemble build` Build Phase:
```
/bin/sh "$FLUTTER_ROOT/packages/flutter_tools/bin/macos_assemble.sh" build
```
Paste this code into the second `macos_assemble embed` Build Phase:
```
/bin/sh "$FLUTTER_ROOT/packages/flutter_tools/bin/macos_assemble.sh" embed
```
Xcode by default also enables a "parallel execution" setting, which
breaks Patrol. Disable it **for all schemes** (if you have more than one):
Go to **RunnerUITests** -> **Build Settings**, search for **User Script Sandboxing**
and make sure it's set to **No**.
Go to **Runner** -> **Signing & Capabilities**. Make sure that in all **App Sandbox**
sections, **Incoming Connections (Server)** and **Outgoing Connections (Client)** checkboxes
are checked.
**Copy** `DebugProfile.entitlements` and `Release.entitlements` files from `macos/Runner`
to `macos/RunnerUITests` directory.
Go to **RunnerUITests** -> **Build Settings** and set **Code Signing Entitlements** to
`RunnerUITests/DebugProfile.entitlements` for **Debug** and **Profile** configuration and to
`RunnerUITests/Release.entitlements` for **Release** configuration.
Create a simple integration test
Let's create a dummy Flutter integration test that you'll use to verify
that Patrol is correctly set up.
Paste the following code into `patrol_test/example_test.dart`:
```dart title="patrol_test/example_test.dart"
import 'dart:io';
import 'package:flutter/material.dart';
import 'package:flutter_test/flutter_test.dart';
import 'package:patrol/patrol.dart';
void main() {
patrolTest(
'counter state is the same after going to home and switching apps',
($) async {
// Replace later with your app's main widget
await $.pumpWidgetAndSettle(
MaterialApp(
home: Scaffold(
appBar: AppBar(title: const Text('app')),
backgroundColor: Colors.blue,
),
),
);
expect($('app'), findsOneWidget);
if (!Platform.isMacOS) {
await $.platform.mobile.pressHome();
}
},
);
}
```
It does only 2 things:
* first, it finds a text `app`
* then (on mobile platforms), it exits to home screen
It's a very simple test, but it's enough to verify that Patrol is correctly set
up. To run `patrol_test/example_test.dart` on a connected Android, iOS or macOS device:
```
patrol test -t patrol_test/example_test.dart
```
If the setup is successful, you should see a summary like one below.
```
Test summary:
📝 Total: 1
✅ Successful: 1
❌ Failed: 0
⏩ Skipped: 0
📊 Report:
⏱️ Duration: 4s
```
If something went wrong, please proceed to the [FAQ] section which might
contain an answer to your issue.
If you are looking for a working example of a Flutter app with Patrol tests,
check out the [example app]
in the patrol repository.
Add test\_bundle.dart to .gitignore
```
If you are using a custom test directory, replace `patrol_test` with your custom directory.
patrol_test/test_bundle.dart
```
This file is generated by Patrol and should not be committed.
## Initializing app inside a test [#initializing-app-inside-a-test]
To be able to test your app, you need to initialize it and pump the app's root widget, so it appears on the screen.
It's very similar to what is done in main function of your app, but it has some key differences, that can break your tests.
Easy way to implement it is to copy main function of your app and then adjust it, so it works with Patrol.
Here's what to remove when running app inside a patrol test:
1. DO NOT call `WidgetsFlutterBinding.ensureInitialized()`.
2. DO NOT use `runApp()`. Instead, use `$.pumpWidget()` (or `$.pumpWidgetAndSettle()` to wait until the UI is rendered). Pass the same argument which was passed to `runApp()`.
3. DO NOT modify `FlutterError.onError`. Sometimes it is done by some monitoring tools (like Crashlytics). Those tools rely on intercepting errors by modifying `FlutterError.onError` callback and it causes that the test engine can't see any exceptions, thus can't end a test if it fails. One way is to move the code that would be common for both the test and the app into a method and leave the rest in main function of the app, or move whole app initialization to a function and define some arguments to enable or diable parts needed in a specific place.
For an example, look at `createApp` in [`common.dart`] in Patrol repository on GitHub.
## Flavors [#flavors]
If your app is using flavors, then you can pass them like so:
```
patrol test --target patrol_test/example_test.dart --flavor development
```
or you can specify them in `pubspec.yaml` (recommended):
```yaml title="pubspec.yaml"
patrol:
app_name: My App
flavor: development
android:
package_name: com.example.myapp
ios:
bundle_id: com.example.MyApp
app_name: The Awesome App
macos:
bundle_id: com.example.macos.MyApp
```
## FAQ [#faq]
The reason is probably a mismatch of `patrol` and `patrol_cli` versions. Go to [Compatibility table]
and make sure that the versions of `patrol` and `patrol_cli` you are using are compatible.
To run your application within the patrol test, you need to call `$.pumpWidgetAndSettle()`,
and pass your application's main widget to it. Be sure that you registered all the
necessary services before calling `$.pumpWidgetAndSettle()`.
Here's the example of running an app within the patrol test:
```dart
void main() {
patrolTest('real app test', ($) async {
// Do all the necessary setup here (DI, services, etc.)
await $.pumpWidgetAndSettle(const MyApp()); // Your's app main widget
// Start testing your app here
});
}
```
It's a good practice to create a setup wrapper function for your tests, so you don't have to
repeat the same code in every test. Look at the [example]
of a wrapper function. Find out more in [Initializing app inside a test] section above.
### Android [#android]
Go to `android/app/build.gradle` and look for `applicationId` in `defaultConfig` section.
It's most likely caused by using incompatible JDK version.
Run `javac -version` to check your JDK version. Patrol officially works on JDK 17,
so unexpected errors may occur on other versions.
If you have AndroidStudio or Intellij IDEA installed, you can find the path to JDK by
opening your project's android directory in AS/Intellij and going to
**Settings** -> **Build, Execution, Deployment** -> **Build Tools** -> **Gradle** -> **Gradle JDK**.
[Learn more]
If the build hangs and running with `--verbose` shows it stuck at `$ flutter build apk --config-only`, updating orchestrator to `1.6.1` in your `build.gradle` might help.
### iOS [#ios]
For iOS go to `ios/Runner.xcodeproj/project.pbxproj` and look for `PRODUCT_BUNDLE_IDENTIFIER`.
For macOS go to `macos/Runner.xcodeproj/project.pbxproj` and look for `PRODUCT_BUNDLE_IDENTIFIER`.
Make sure that you disabled "Paralell execution" for **all schemes** in Xcode.
See [this video] for details.
Search for a `FLUTTER_TARGET` in your project files and remove it (both value and key)
from \*.xcconfig and \*.pbxproj files.
Search for a `FLUTTER_TARGET` in your project files and remove it (both value and key)
from \*.xcconfig and \*.pbxproj files.
Check if this line in `Podfile` is present and uncommented.
```
platform :ios, '12.0'
```
If yes, then check if **iOS deployment version** in Xcode project's **Build Settings**
section for all targets (Runner and RunnerUITests) are set to the same value as in Podfile
(in case presented in snippet above, all should be set to 12.0).
After removing `FLUTTER_TARGET` from `*.xcconfig`, you need to run the following command to generate it:
```
flutter build ios --config-only patrol_test/example_test.dart
```
Make sure to replace `patrol_test/example_test.dart` with the path to your test file.
If you couldn't find an answer to your question/problem, feel free to ask on
[Patrol Discord Server].
## Going from here [#going-from-here]
To learn how to write Patrol tests, see [finders] and [native automation] sections.
[native automation]: /documentation/native/usage
[Setup for physical iOS Devices]: /documentation/physical-ios-devices-setup
[finders]: /documentation/finders/usage
[Using Patrol finders in widget tests]: /documentation/finders/finders-setup
[Here's why]: /documentation/native/advanced#embrace-the-native-tests
[Patrol CLI]: https://pub.dev/packages/patrol_cli
[FAQ]: /documentation#faq
[Compatibility table]: /documentation/compatibility-table
[README]: https://pub.dev/packages/patrol_cli#installation
[example app]: https://github.com/leancodepl/patrol/tree/master/packages/patrol/example
[example]: https://github.com/leancodepl/patrol/blob/d2c7493f9399a028e39cb94fd204affdb932c5fc/dev/e2e_app/patrol_test/common.dart#L17-L33
[Learn more]: https://developer.android.com/build/jdks
[this video]: https://www.youtube.com/watch?v=9LdEJR59fW4
[Patrol Discord Server]: https://discord.gg/ukBK5t4EZg
[Initializing app inside a test]: /documentation#initializing-app-inside-a-test
[`common.dart`]: https://github.com/leancodepl/patrol/blob/master/dev/e2e_app/integration_test/common.dart
# Logs and test results
Once you've written and executed your tests, it's essential to monitor their results. Patrol provides two main methods for reporting test outcomes: **console logs** and **native test reports**.
## Logging test steps [#logging-test-steps]
This feature is available starting from version `3.13.0`.
If you're using this version but don't see logs for test steps, check if you're passing a custom `PatrolTesterConfig` to `patrolTest()`. If so, ensure the `printLogs: true` argument is included in the constructor.
During test execution, every test step (e.g., `tap` or `enterText`) is logged to the console along with its status. Additionally, the test name, status, and execution time are displayed.
**Example console output:**
```
...
🧪 denies various permissions
✅ 1. scrollTo widgets with text "Open permissions screen".
✅ 2. scrollTo widgets with text "Open permissions screen".
✅ 3. tap widgets with text "Open permissions screen".
✅ 4. tap widgets with text "Request camera permission".
✅ 5. isPermissionDialogVisible (native)
✅ 6. tap widgets with text "Request camera permission".
✅ 7. isPermissionDialogVisible (native)
⏳ 8. denyPermission (native)
❌ denies various permissions (patrol_test/permissions/deny_many_permissions_twice_test.dart) (9s)
══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞═════════════════
The following PlatformException was thrown running a test:
PlatformException(PermissionHandler.PermissionManager, A request
for permissions is already running, please wait for it to finish
before doing another request (note that you can request multiple
permissions at the same time)., null, null)
When the exception was thrown, this was the stack:
#0 StandardMethodCodec.decodeEnvelope (package:flutter/src/services/message_codecs.dart:648:7)
#1 MethodChannel._invokeMethod (package:flutter/src/services/platform_channel.dart:334:18)
#2 MethodChannelPermissionHandler.requestPermissions (package:permission_handler_platform_interface/src/method_channel/method_channel_permission_handler.dart:79:9)
#3 PermissionActions.request (package:permission_handler/permission_handler.dart:52:31)
#4 _PermissionsScreenState._requestCameraPermission (package:e2e_app/permissions_screen.dart:21:20)
The test description was:
denies various permissions
═════════════════════════════════════════════════════════════════
✅ taps on notification (patrol_test/permissions/notifications_test.dart) (16s)
✅ taps on notification native2 (patrol_test/permissions/notifications_test.dart) (14s)
✅ grants various permissions (patrol_test/permissions/permissions_many_test.dart) (15s)
...
```
## Test summary [#test-summary]
Once the tests are complete, a summary is printed:
```
Test summary:
📝 Total: 8
✅ Successful: 3
❌ Failed: 5
- taps on notification (patrol_test/permissions/notifications_test.dart)
- taps on notification native2 (patrol_test/permissions/notifications_test.dart)
- accepts location permission (patrol_test/permissions/permissions_location_test.dart)
- accepts location permission native2 (patrol_test/permissions/permissions_location_test.dart)
- grants various permissions (patrol_test/permissions/permissions_many_test.dart)
⏩ Skipped: 0
📊 Report: file:///Users/user/patrol/dev/e2e_app/build/app/reports/androidTests/connected/index.html
⏱️ Duration: 227s
```
## Customizing log behavior [#customizing-log-behavior]
You can customize which logs are displayed by using the following flags. These can be passed to the `patrol test` or `patrol develop` commands:
| Flag | Description | Available in | Default value |
| ------------------------- | ------------------------------------------ | ------------------------------------------------- | ------------- |
| --\[no-]show-flutter-logs | Show Flutter logs while running the tests. | `patrol test`, in `patrol develop` it's always on | `false` |
| --\[no-]hide-test-steps | Hide test steps while running the tests. | `patrol test` and `patrol develop` | `false` |
| --\[no-]clear-test-steps | Clear test steps after the test finishes. | `patrol test` | `true` |
## Native test reports [#native-test-reports]
In addition to console logs, you can review test results in a **native test report**. The report's file path is provided in the test summary, for example:
### Android: [#android]
Path without flavor, so just build mode (debug, release, profile):
```
📊 Report: file:///Users/user/patrol/dev/e2e_app/build/app/reports/androidTests/connected/debug/index.html
```
Path with flavor:
```
📊 Report: file:///Users/user/patrol/dev/e2e_app/build/app/reports/androidTests/connected/debug/flavors/dev/index.html
```
### iOS: [#ios]
Open `.xcresult` file in Xcode, to see logs and videos.
Path:
```
📊 Report: file:///Users/user/patrol/dev/e2e_app/build/ios_results_1754923033891.xcresult
```
## Logs in `patrol_finders` [#logs-in-patrol_finders]
By default, enhanced logs are disabled when using `patrol_finders` without the `patrol` package. To enable them, pass the `printLogs: true` argument to the `PatrolTesterConfig` constructor:
```dart
patrolWidgetTest(
'throws exception when no widget to tap on is found',
config: const PatrolTesterConfig(printLogs: true),
(tester) async {
// test body
// ...
},
);
```
```dart
testWidgets(
'description',
(widgetTester) async {
final $ = PatrolTester(
tester: widgetTester,
config: PatrolTesterConfig(printLogs: true),
);
// test body
// ...
},
);
```
# Physical iOS devices
Because of restrictions on JIT, "in iOS 14+, debug mode Flutter apps can only be launched from Flutter tooling,IDEs..." we need to build and run tests in release mode.
Going further, we need to sign the app and tests. Let's assume that your team joined Apple Developer Program and have already an App ID, profile and certificate for your app. We need to do the same for `RunnerUITests`.
In [Apple Developer Portal](https://developer.apple.com) create a new Identifier (App ID) with your app's bundle ID and with `.RunnerUITests.xctrunner` ending. It should look like e.g.: `com.example.myapp.RunnerUITests.xctrunner` (Remember to swap `com.example.myapp` with your app's one).
In [Apple Developer Portal](https://developer.apple.com) make sure that you have a Development Certificate. If not, create one.
In [Apple Developer Portal](https://developer.apple.com) create a new Development Provisioning Profile that is linked to the new Identifier and includes the Development Certificate.
Next three steps are needed only if you use [fastlane for codesigning](https://docs.fastlane.tools/codesigning/getting-started/).
In Xcode disable automatic signing in `RunnerUITests` Target.
Go to `ios/Runner.xcodeproj/project.pbxproj` and search for all occurrences of `PRODUCT_BUNDLE_IDENTIFIER = com.example.myapp.RunnerUITests;`. Then set the `PROVISIONING_PROFILE_SPECIFIER` to your specifier of newly created profile.
Import the new profile for RunnerUITests manually in Xcode.
You might see error or warning in Xcode about not matching bundle IDs. Don't worry about it, `RunnerUITests.xctrunner` bundle is generated while building tests and this is the bundle ID that we need to sign.
Go to your IDE and try it out by running `patrol build ios --release`. If tests have been built successfully you're ready for action!
When running tests on a physical iOS device, a failed test can cause `xcodebuild` to hang at a `password:` prompt while collecting diagnostics. The command will remain blocked until interrupted (e.g., by pressing `Ctrl+C`).
To avoid this, if you are using a Test Plan, add `"diagnosticCollectionPolicy": "Never"` to the `defaultOptions` section in your `.xctestplan` file (e.g., `ios/TestPlan.xctestplan`):
```json
{
"defaultOptions" : {
"diagnosticCollectionPolicy" : "Never"
}
}
```
# Supported platforms
Patrol works on:
* Android 5.0 (API 21) and newer,
* iOS 13 and newer,
* macOS 10.14 and newer (alpha support).
Windows and Linux are not supported.
On mobile platforms it works on both physical and virtual devices.
If you want to check which native features are supported, see [feature parity].
[feature parity]: /documentation/native/feature-parity
# Flutter Web Testing
Patrol provides support for running integration tests on Flutter web applications using
[Playwright](https://playwright.dev/), a powerful browser automation framework. This allows you to handle native web
browser interactions in your tests.
## How it Works [#how-it-works]
When you run Patrol tests on web, the following happens:
1. **Flutter Web Server**: Patrol starts a Flutter web server that serves your application
2. **Playwright Test Runner**: Patrol automatically launches Chromium using Playwright
3. **Platform Actions Bridge**: For platform automation calls (like `$.platform.web.grantPermissions()`), your Dart test code sends requests to Playwright with actions to be executed
4. **Results Collection**: Test results are collected and reported back in Patrol's standard format
This architecture allows you to write the same Patrol tests that work across mobile and web platforms, with Playwright handling browser-specific interactions.
## Prerequisites [#prerequisites]
Before running Patrol tests on web, ensure you have installed [Node.js](https://nodejs.org/).
## Running Tests on Web [#running-tests-on-web]
To run your Patrol tests on web, use the `--device` flag with `chrome`, e.g.:
```bash
patrol test --device chrome --target patrol_test/login_test.dart
```
When you first run tests on web, Patrol will automatically install Node.js dependencies
including Playwright and its browser binaries. This may take a moment.
### Running Tests in your CI pipeline [#running-tests-in-your-ci-pipeline]
CI environments typically do not provide a graphical display. For this reason, it's recommended to run Patrol web tests in **headless mode**.
By default, Patrol runs web tests in headed mode (with a visible browser window).\
In CI pipelines (e.g. GitHub Actions, GitLab CI), it's recommended to enable headless mode using `--web-headless true`:
```bash
patrol test \
--device chrome \
--target patrol_test/login_test.dart \
--web-headless true
```
### Supported Native Actions on Web [#supported-native-actions-on-web]
Patrol supports the following platform actions through Playwright:
```dart
// Dark mode
await $.platform.web.enableDarkMode();
await $.platform.web.disableDarkMode();
// Keyboard input
await $.platform.web.pressKey(key: 'a');
await $.platform.web.pressKeyCombo(keys: ['Control', 'a']);
// Permissions
await $.platform.web.grantPermissions(permissions: ['clipboard-read', 'clipboard-write']);
await $.platform.web.clearPermissions();
// Clipboard
await $.platform.web.setClipboard(text: 'test');
final clipboard = await $.platform.web.getClipboard();
// Browser navigation
await $.platform.web.goBack();
await $.platform.web.goForward();
// Cookies
await $.platform.web.addCookie(name: 'test_cookie', value: 'cookie_value', url: 'http://localhost:8080');
final cookies = await $.platform.web.getCookies();
await $.platform.web.clearCookies();
// File operations
final fileContent = utf8.encode('Hello from Patrol file upload test!');
final file = UploadFileData(
name: 'example_file.txt',
content: fileContent,
mimeType: 'text/plain',
);
await $.platform.web.uploadFile(files: [file]);
final downloads = await $.platform.web.verifyFileDownloads();
// Dialogs - because of blocking nature of dialogs, dialogs need to be subscribed to before triggering
final dialogFuture = $.platform.web.acceptNextDialog();
// ... trigger the dialog ...
final dialogText = await dialogFuture;
final dialogFuture = $.platform.web.dismissNextDialog();
// ... trigger the dialog ...
final dialogText = await dialogFuture;
// Web element interactions (for iframes and external HTML)
await $.platform.web.scrollToWeb(selector: inputSelector, iframeSelector: iframeSelector);
await $.platform.web.enterTextWeb(selector: inputSelector, text: 'Hello from Patrol!', iframeSelector: iframeSelector);
await $.platform.web.tapWeb(selector: buttonSelector, iframeSelector: iframeSelector);
// Window operations
await $.platform.web.resizeWindow(size: Size(800, 600));
```
# Write your first test
Patrol is a powerful, open-source testing framework created by LeanCode that enhances Flutter's testing capabilities by enabling interaction with native platform features directly in Dart. It allows to handle permission dialogs, notifications, WebViews, and device settings—features previously unavailable in standard Flutter tests, making it truly possible to test the whole app.
This tutorial will take you through writing your first substantial Patrol test, interacting both with the Flutter app itself and also with native permission dialogs and notifications.
Before writing any tests, make sure you [install the Patrol CLI]. Then just clone the following repository from GitHub to follow along. The app we’re going to be testing is fully functional and ready to be tested, with Patrol already configured.
To learn how to set up Patrol for your own project, check out the [Patrol Setup Docs].
Clone the [STARTER PROJECT] to follow along.
### App Walkthrough [#app-walkthrough]
Before we can start writing automated Patrol tests, we need to know what the app does and to test it manually. Please, check out the video tutorial for a visual walkthrough.
The first screen of our app is for signing in. It’s not using any actual sign-in provider but it only validates the email address and password. In order to successfully “sign in” and get to the home screen, we need to input a valid email and a password that’s at least 8 characters long.
You can test any real authentication providers that use WebView for signing in with the powerful Patrol native automation.
On the second screen, we’re greeted with a notifications permission dialog. Once we allow them, we can tap on the notification button in the app bar to manually trigger a local notification which will be displayed after 3 seconds both when the app is running in the foreground or in the background.
Once we open the native notification bar and tap on the notification from our app, we’re gonna see a snackbar on the bottom saying *"Notification was tapped!”*
### Testing the “Happy Path” [#testing-the-happy-path]
You’ve just seen the full walkthrough of the app, including errors that can show up if you input an invalid email or password. UI tests (integration tests), like the ones we’re going to write with Patrol, should only be testing the “happy path” of a UI flow. We only want them to fail if the app suddenly stops the user from doing what the app is for - in this case, that’s displaying a notification. Validation error messages are not “what the app is for”, they exist only to allow the user to successfully sign in with a proper email and password. That’s why we won’t be checking for them in the tests.
### Writing the Test [#writing-the-test]
We have only one UI flow in this app, that is signing in, showing the notification and then tapping on that notification. This means, we’re going to have only a single test. Let’s create it in `/patrol_test/app_test.dart`.
Like any other test, we need to have a `main()` top-level function. Inside it we’re going to have our single `patrolTest` with a description properly describing what we’re about to test. An optional step is to set the frame policy to “fully live” to make sure all frames are shown, even those which are not pumped by our test code. Without it, we would see that our app stutters and animations are not played properly.
```dart
import 'package:flutter_test/flutter_test.dart';
import 'package:patrol/patrol.dart';
void main() {
patrolTest(
'signs in, triggers a notification, and taps on it',
($) async {
// Test code will go here
},
);
}
```
We could start writing the test right now and then re-run it from scratch every time we add a new line of test code by calling `patrol test --target patrol_test/app_test.dart` but since we’re writing a Patrol test that runs on an Android or iOS device, constantly building the whole Flutter app is not time effective. Thankfully, Patrol offers a different approach - hot restarting the tests! We can run the command `patrol develop --target patrol_test/app_test.dart` right now and anytime we add a new line of test code, we can just type “r” in the terminal to re-run the tests without the time-costly app building. Just make sure that you have an emulator running first - Patrol will select it automatically.
First, we need to perform any initializations that need to happen before the app is run and pump the top-level widget of our app. We’re effectively doing what the `main` function inside `main.dart` does - this time not for just running the app as usual but for running an automated Patrol test.
```dart
patrolTest(
'signs in, triggers a notification, and taps on it',
($) async {
initApp();
await $.pumpWidgetAndSettle(const MainApp());
},
);
```
Hot-restarting the test by typing “r” into the terminal won’t really do much since we’re not yet performing any user-like actions but you will at least see the sign in page for a brief moment before the test finishes.
Let’s now perform some action! We know we have to sign in if we want to continue to the home screen. First, we have to type in both email and password. There are multiple ways to find widgets on the screen - by widget type, by text and lastly by key.
Although it’s not the best practice, we’re first going to find the fields by type. Both are of type `TextFormField` but there are two of them on the screen so the following won’t work.
```dart
patrolTest(
'signs in, triggers a notification, and taps on it',
($) async {
initApp();
await $.pumpWidgetAndSettle(const MainApp());
await $(TextFormField).enterText('test@email.com');
await $(TextFormField).enterText('password');
},
);
```
That’s because finders always find the first matching widget so both the email address and password are entered into the same field - in this case, the email field.
If multiple widgets on a screen match the finder, we can tell Patrol which one we want by specifying its index in the list of all found widgets from top to bottom like this:
```dart
patrolTest(
'signs in, triggers a notification, and taps on it',
($) async {
initApp();
await $.pumpWidgetAndSettle(const MainApp());
await $(TextFormField).enterText('test@email.com');
await $(TextFormField).at(1).enterText('password');
},
);
```
We can use a text finder to tap on the “Sign in” button.
```dart
patrolTest(
'signs in, triggers a notification, and taps on it',
($) async {
initApp();
await $.pumpWidgetAndSettle(const MainApp());
await $(TextFormField).enterText('test@email.com');
await $(TextFormField).at(1).enterText('password');
await $('Sign in').tap();
},
);
```
Hot-restarting the test will now take you all the way to the home page from which we will want to trigger the notification.
As you can imagine though, using type and text finders in any app that’s just a bit more complex will result in a huge mess. The recommended approach is to always find your widgets by their `Key`. There are currently no keys specified for these widgets so let’s change that. In `sign_in_page.dart` pass in the following into the `TextFormFields` and `ElevatedButton`:
```dart
class SignInPage extends StatelessWidget {
const SignInPage({super.key});
@override
Widget build(BuildContext context) {
return Scaffold(
...
body: Padding(
padding: const EdgeInsets.all(16),
child: Form(
child: Column(
mainAxisAlignment: MainAxisAlignment.center,
children: [
TextFormField(
key: const Key('emailTextField'),
decoration: const InputDecoration(
labelText: 'Email',
),
...
),
const SizedBox(height: 16),
TextFormField(
key: const Key('passwordTextField'),
decoration: const InputDecoration(
labelText: 'Password',
),
...
),
const SizedBox(height: 16),
Builder(builder: (context) {
return ElevatedButton(
key: const Key('signInButton'),
...
child: const Text('Sign in'),
);
}),
],
),
),
),
);
}
}
```
With the keys in place, we can now rewrite our test code to use `Key` finders. The simplest approach is to prefix the key’s value with a hash symbol. For this approach to work, your keys mustn’t contain any invalid characters such as spaces.
```dart
patrolTest(
'signs in, triggers a notification, and taps on it',
($) async {
initApp();
await $.pumpWidgetAndSettle(const MainApp());
await $(#emailTextField).enterText('test@email.com');
await $(#passwordTextField).enterText('password');
await $(#signInButton).tap();
},
);
```
Looking at this test code again, it’s certain we can do better. Why? We’ve just added code duplication to our codebase! The key values in `sign_in_page.dart` and in `app_test.dart` are fully duplicated and if we change one, the other won’t be automatically updated, thus breaking our tests.
That’s why production-grade apps should have a single source for all the `Keys` exposed as a global final variable inside `integration_test_keys.dart`. That’s going to look as follows if we already take into account the home page which we want to test next.
```dart
import 'package:flutter/foundation.dart';
class SignInPageKeys {
final emailTextField = const Key('emailTextField');
final passwordTextField = const Key('passwordTextField');
final signInButton = const Key('signInButton');
}
class HomePageKeys {
final notificationIcon = const Key('notificationIcon');
final successSnackbar = const Key('successSnackbar');
}
class Keys {
final signInPage = SignInPageKeys();
final homePage = HomePageKeys();
}
final keys = Keys();
```
Feel free to put your page-specific key classes (e.g. `SignInPageKeys`) into separate files in more complex apps.
The updated `sign_in_page.dart` code will now look like this:
```dart
class SignInPage extends StatelessWidget {
const SignInPage({super.key});
@override
Widget build(BuildContext context) {
return Scaffold(
...
body: Padding(
padding: const EdgeInsets.all(16),
child: Form(
child: Column(
mainAxisAlignment: MainAxisAlignment.center,
children: [
TextFormField(
key: keys.signInPage.emailTextField,
decoration: const InputDecoration(
labelText: 'Email',
),
...
),
const SizedBox(height: 16),
TextFormField(
key: keys.signInPage.passwordTextField,
decoration: const InputDecoration(
labelText: 'Password',
),
...
),
const SizedBox(height: 16),
Builder(builder: (context) {
return ElevatedButton(
key: keys.signInPage.signInButton,
...
child: const Text('Sign in'),
);
}),
],
),
),
),
);
}
}
```
The test code will now also use the `keys` global final variable instead of the hash symbol notation:
```dart
patrolTest(
'signs in, triggers a notification, and taps on it',
($) async {
initApp();
await $.pumpWidgetAndSettle(const MainApp());
await $(keys.signInPage.emailTextField).enterText('test@email.com');
await $(keys.signInPage.passwordTextField).enterText('password');
await $(keys.signInPage.signInButton).tap();
},
);
```
Hot-restarting the test won’t show any change in its functionality but it sure is more maintainable and easier to work with.
### Home Page [#home-page]
First, let’s add the keys we’ve already created to the `IconButton` and the `SnackBar` shown when the notification has been tapped.
```dart
class HomePage extends StatefulWidget {
...
}
class _HomePageState extends State {
...
@override
Widget build(BuildContext context) {
return Scaffold(
appBar: AppBar(
title: const Text('Home'),
actions: [
IconButton(
key: keys.homePage.notificationIcon,
icon: const Icon(Icons.notification_add),
onPressed: () {
triggerLocalNotification(
onPressed: () {
ScaffoldMessenger.of(context).showSnackBar(
SnackBar(
key: keys.homePage.successSnackbar,
content: const Text('Notification was tapped!'),
),
);
},
onError: () {
...
},
);
},
),
],
),
...
);
}
}
```
The first thing the user sees when first navigating to the `HomePage` is a notifications permission dialog. We need to accept it from within the test. Patrol’s native automation makes this as easy as it gets.
Native automation allows you to interact with the OS your Flutter app is running on. Patrol currently supports Android, iOS and macOS native interactions. [Learn more from the docs]
```dart
patrolTest(
'signs in, triggers a notification, and taps on it',
($) async {
initApp();
await $.pumpWidgetAndSettle(const MainApp());
await $(keys.signInPage.emailTextField).enterText('test@email.com');
await $(keys.signInPage.passwordTextField).enterText('password');
await $(keys.signInPage.signInButton).tap();
await $.platform.mobile.grantPermissionWhenInUse();
},
);
```
Hot-restarting the test will work wonderfully the first time, however, once the permission has already been granted, calling `grantPermissionWhenInUse()` will fail. This is not going to be an issue if you use Patrol as a part of your CI/CD process since everytime you test with Patrol there, the app will be built from scratch and no permission will be granted yet. But when we’re writing the test locally with `patrol develop` command, we need to make sure that the permission dialog is visible before trying to accept it.
```dart
patrolTest(
'signs in, triggers a notification, and taps on it',
($) async {
initApp();
await $.pumpWidgetAndSettle(const MainApp());
await $(keys.signInPage.emailTextField).enterText('test@email.com');
await $(keys.signInPage.passwordTextField).enterText('password');
await $(keys.signInPage.signInButton).tap();
if (await $.platform.mobile.isPermissionDialogVisible()) {
await $.platform.mobile.grantPermissionWhenInUse();
}
},
);
```
It’s generally a bad practice to add any branching logic within your tests and you should be 100% certain that it cannot introduce any test flakiness before doing so. Checking if a permission dialog is visible is an example of a proper use of branching logic.
Next up, we want to tap on the notification icon button and then go to the device home screen to test the notification while the app is running in the background.
```dart
patrolTest(
'signs in, triggers a notification, and taps on it',
($) async {
initApp();
await $.pumpWidgetAndSettle(const MainApp());
await $(keys.signInPage.emailTextField).enterText('test@email.com');
await $(keys.signInPage.passwordTextField).enterText('password');
await $(keys.signInPage.signInButton).tap();
if (await $.platform.mobile.isPermissionDialogVisible()) {
await $.platform.mobile.grantPermissionWhenInUse();
}
await $(keys.homePage.notificationIcon).tap();
await $.platform.mobile.pressHome();
},
);
```
Once we’re on the home screen, we want to open the notification shade and tap on the notification we get from our app. You can either tap on a notification by index or by finding a text. We know that the title of our notification is “Patrol says hello!” so let’s do the latter.
```dart
patrolTest(
'signs in, triggers a notification, and taps on it',
($) async {
initApp();
await $.pumpWidgetAndSettle(const MainApp());
await $(keys.signInPage.emailTextField).enterText('test@email.com');
await $(keys.signInPage.passwordTextField).enterText('password');
await $(keys.signInPage.signInButton).tap();
if (await $.platform.mobile.isPermissionDialogVisible()) {
await $.platform.mobile.grantPermissionWhenInUse();
}
await $(keys.homePage.notificationIcon).tap();
await $.platform.mobile.pressHome();
await $.platform.mobile.openNotifications();
await $.platform.mobile.tapOnNotificationBySelector(
Selector(textContains: 'Patrol says hello!'),
);
},
);
```
Since the notification is delayed by 3 seconds, we have to provide a timeout that’s at least as long in order to wait for the notification to appear - 5 seconds should do the trick here.
```dart
patrolTest(
'signs in, triggers a notification, and taps on it',
($) async {
...
await $.platform.mobile.openNotifications();
await $.platform.mobile.tapOnNotificationBySelector(
Selector(textContains: 'Patrol says hello!'),
timeout: const Duration(seconds: 5),
);
},
);
```
Lastly, we want to check if the snackbar has been shown after tapping on a notification. We can call `waitUntilVisible()` after selecting it with its key.
```dart
patrolTest(
'signs in, triggers a notification, and taps on it',
($) async {
initApp();
await $.pumpWidgetAndSettle(const MainApp());
await $(keys.signInPage.emailTextField).enterText('test@email.com');
await $(keys.signInPage.passwordTextField).enterText('password');
await $(keys.signInPage.signInButton).tap();
if (await $.platform.mobile.isPermissionDialogVisible()) {
await $.platform.mobile.grantPermissionWhenInUse();
}
await $(keys.homePage.notificationIcon).tap();
await $.platform.mobile.pressHome();
await $.platform.mobile.openNotifications();
await $.platform.mobile.tapOnNotificationBySelector(
Selector(textContains: 'Patrol says hello!'),
timeout: const Duration(seconds: 5),
);
$(keys.homePage.successSnackbar).waitUntilVisible();
},
);
```
And just like that, we have now tested the whole flow of the app with Patrol! If any part of the logic breaks, this test will notify us about that sooner than our real users do and that’s what we’re all after!
[install the Patrol CLI]: https://pub.dev/packages/patrol_cli#installation
[Patrol Setup Docs]: /documentation
[STARTER PROJECT]: https://github.com/ResoCoder/patrol-basics-tutorial
[Learn more from the docs]: /documentation/native/overview
# Disabling/enabling Bluetooth
In this video we show you how to toggle Bluetooth using Patrol framework.
Here you can find the code of this test and try it out by yourself.
```dart title="patrol_test/bluetooth_test.dart"
import 'package:example/main.dart';
import 'package:patrol/patrol.dart';
void main() {
patrolTest(
'disable and enable bluetooth',
($) async {
await $.pumpWidgetAndSettle(const MyApp());
await Future.delayed(const Duration(seconds: 2));
await $.platform.mobile.openQuickSettings();
await $.platform.mobile.disableBluetooth();
await Future.delayed(const Duration(seconds: 4));
await $.platform.mobile.enableBluetooth();
await Future.delayed(const Duration(seconds: 4));
},
);
}
```
# Granting camera permission
In this video we show you how to grant camera permission using Patrol framework.
Here you can find the code of this test and try it out by yourself.
```dart title="patrol_test/grant_camera_permission_test.dart"
import 'package:permission_handler/permission_handler.dart';
import './common.dart';
void main() {
patrolTest('grants camera permission', ($) async {
await createApp($);
await $('Open permissions screen').scrollTo().tap();
if (!await Permission.camera.isGranted) {
await $('Request camera permission').tap();
if (await $.platform.mobile.isPermissionDialogVisible()) {
await Future.delayed(const Duration(seconds: 1));
await $.platform.mobile.grantPermissionWhenInUse();
await $.pump();
}
}
await Future.delayed(const Duration(seconds: 4));
});
}
```
# Introduction
Welcome to the **Patrol Feature Guide — your go-to resource for getting the most out of Patrol**, the powerful UI testing framework for Flutter apps. Whether you're a Flutter developer, QA engineer, or tech lead, this guide will help you understand how Patrol can supercharge your automated testing workflow.
In this guide, you’ll find a breakdown of **Patrol’s core features, usage examples, and best practices** to help you write stable, maintainable, and meaningful tests. Whether you're just getting started or looking to go deeper, this documentation will support you every step of the way.
That’s not all — there are more Patrol features than you see on the list on the left! We’re working on a series of videos to show you how to make the most of each feature in action. Stay tuned!
If you haven’t used Patrol yet, **remember that you need to set it up first**. Your go-to place is [this page in our documentation].
**Need expert help?** LeanCode offers end-to-end automated UI testing services tailored for your Flutter apps – [check them out here].
[this page in our documentation]: /documentation
[check them out here]: https://leancode.co/products/automated-ui-testing-in-flutter?utm_source=patrol_page&utm_medium=link&utm_campaign=service
# Pick images from gallery
Patrol provides functionality to pick one or many images from the Android and iOS gallery using the `pickImageFromGallery` or `pickMultipleImagesFromGallery` methods from the native automator.
Due to differences between devices and gallery apps, this method may not work on 100% of devices, but it should work on most. If the device you are testing on is not working with the default selectors, you can provide your own selectors.
To get native selectors, you can use the [Patrol DevTools Extension](https://patrol.leancode.co/documentation/patrol-devtools-extension).
## Pick an image from gallery [#pick-an-image-from-gallery]
This method performs the following actions:
1. Selects an image (by default, the first image or the one at the provided index).
2. Confirm the selection if needed (some android devices require this step).
| Platform | Selector (default) |
| -------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| Physical/Emulator Android | `com.google.android.documentsui:id/icon_thumb` (API \< 34) or `com.google.android.providers.media.module:id/icon_thumbnail` (API 34+) |
| Simulator and Physical iOS | `IOSElementType.image` (we add +2 for simulators and +1 for physical devices to the index, as image finders start from index 1 or 2 depending on the device) |
This method should work without custom selectors on iOS devices. For Android, it should work with most emulators and Pixel physical devices.
### Examples [#examples]
When you are using supported devices, you can use this method without any additional arguments:
```dart
import 'package:example/main.dart';
import 'package:patrol/patrol.dart';
void main() {
patrolTest(
'Pick an image from gallery on Android or iOS',
($) async {
await $.pumpWidgetAndSettle(const MyApp());
await $.tap(#addPhotoFromGalleryButton); // Opens the gallery picker in your app
await $.platform.mobile.grantPermissionWhenInUse(); // Some devices require permission to be granted before picking an image
await $.platform.mobile.pickImageFromGallery(index: 0); // Picks the first image from the gallery
},
);
}
```
When you are using unsupported devices, you can provide your own selectors:
```dart
import 'package:example/main.dart';
import 'package:patrol/patrol.dart';
void main() {
patrolTest(
'Pick an image from gallery with custom selectors',
($) async {
await $.pumpWidgetAndSettle(const MyApp());
await $.tap(#addPhotoFromGalleryButton); // Opens the gallery picker in your app
await $.platform.mobile.grantPermissionWhenInUse(); // Some devices require permission to be granted before picking an image
await $.platform.mobile.pickImageFromGallery(
index: 1,
imageSelector: NativeSelector(
android: AndroidSelector(
resourceName: 'com.oplus.gallery:id/image',
),
ios: IOSSelector(label: 'Photo'),
),
);
},
);
}
```
## Pick multiple images from gallery [#pick-multiple-images-from-gallery]
This method performs the following actions:
1. Selects multiple images (user needs to specify indexes of images to select).
2. Confirm the selection.
The table below shows the default native selectors that the `pickMultipleImagesFromGallery()` method uses internally for each platform:
| Platform | Selector (default) |
| --------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------- |
| Physical/Emulator Android images | `com.google.android.documentsui:id/icon_thumb` (API \< 34) or `com.google.android.providers.media.module:id/icon_thumbnail` (API 34+) |
| Simulator and Physical iOS images | `IOSElementType.image` |
| iOS Confirm button | `IOSElementType.button` with label `"Add"` |
| Android Confirm button | `com.google.android.providers.media.module:id/button_add` (API 34+) or `com.google.android.documentsui:id/action_menu_select` (API \< 34) |
This method should work without custom selectors on iOS devices. For Android, it should work with most emulators and Pixel physical devices.
### Examples [#examples-1]
When you are using supported devices, you can use this method without any additional arguments:
```dart
import 'package:example/main.dart';
import 'package:patrol/patrol.dart';
void main() {
patrolTest(
'Pick multiple images from gallery',
($) async {
await $.pumpWidgetAndSettle(const MyApp());
await $.tap(#addMultiplePhotosFromGalleryButton); // Opens the gallery picker in your app
await $.platform.mobile.grantPermissionWhenInUse(); // Some devices require permission to be granted before picking an image
await $.platform.mobile.pickMultipleImagesFromGallery(imageIndexes: [0, 1]); // Picks the first and second images from the gallery
},
);
}
```
When you are using unsupported devices, you can provide your own selectors:
```dart
import 'package:example/main.dart';
import 'package:patrol/patrol.dart';
void main() {
patrolTest(
'Pick multiple images from gallery with custom selectors',
($) async {
await $.pumpWidgetAndSettle(const MyApp());
await $.tap(#addMultiplePhotosFromGalleryButton); // Opens the gallery picker in your app
await $.platform.mobile.grantPermissionWhenInUse(); // Some devices require permission to be granted before picking an image
await $.platform.mobile.pickMultipleImagesFromGallery(
imageIndexes: [0, 1],
imageSelector: NativeSelector(
android: AndroidSelector(
resourceName: 'com.oplus.gallery:id/image',
),
ios: IOSSelector(label: 'Photo'),
),
);
},
);
}
```
# Pull to refresh
Patrol provides a function for pull-to-refresh gesture (`pullToRefresh`), allowing you to use refresh functionality in your tests.
## Basic usage [#basic-usage]
The `pullToRefresh` method is available through the platform automator:
```dart
await $.platform.mobile.pullToRefresh();
```
By default, this performs a pull-to-refresh gesture from the center of the screen (0.5, 0.5) to the bottom center (0.5, 0.9).
## Simulating real scenarios [#simulating-real-scenarios]
Sometimes you need to pull to refresh multiple times until a specific element appears. Here's how you can do it:
```dart
const maxAttempts = 5;
var attempt = 0;
while (attempt < maxAttempts) {
// Perform pull to refresh
await $.platform.mobile.pullToRefresh();
// Wait for the refresh to complete
await $.pumpAndSettle();
// Check if the target element exists
if ($(K.awaitedElement).exists) {
break;
}
attempt++;
}
// Verify if the element is visible
await $(K.awaitedElement).waitUntilVisible();
```
## Tips [#tips]
* Increase the `steps` value for a slower gesture
* Since this function is native, it does not benefit from the default `pumpAndSettle` being performed automatically.
* Adjust coordinates based on your app's layout (e.g., when the center of the screen is not part of the scrollable area, or when testing horizontal lists that implement pull-to-refresh)
```dart
// Pull to refresh horizontally (swipe left)
await $.platform.mobile.pullToRefresh(
start: Offset(0.5, 0.5),
end: Offset(0.1, 0.5),
);
```
# Take photo using camera
Patrol provides functionality to take a photo using the Android and iOS camera.
Due to many differences between devices, this method will not work on 100% of devices but should work on most of them. If the device that you are testing on is not working with this command, you can provide your own selectors. To get native selectors, you can use [Patrol DevTools Extension](https://patrol.leancode.co/documentation/patrol-devtools-extension).
## How it works [#how-it-works]
This method does two actions:
1. Tap on shutter button
2. Tap on confirm button
The table below shows the native selectors that the `takeCameraPhoto()` method uses internally for each platform:
| Platform | Shutter Button | Confirm Button |
| -------------------------- | --------------------------------------------------- | ------------------------------------------------ |
| Physical Android | `com.google.android.GoogleCamera:id/shutter_button` | `com.google.android.GoogleCamera:id/done_button` |
| Emulator Android | `com.android.camera2:id/shutter_button` | `com.android.camera2:id/done_button` |
| Simulator and Physical iOS | `PhotoCapture` | `Done` |
This method should work without custom selectors on iOS devices. For Android, it should work with most emulators and Pixel physical devices.
## Examples [#examples]
When you are using supported devices, you can use this method without any additional arguments:
```dart
import 'package:example/main.dart';
import 'package:patrol/patrol.dart';
void main() {
patrolTest(
'Take a photo using android or iOS camera',
($) async {
await $.pumpWidgetAndSettle(const MyApp());
await $.tap(#addPhotoButton); // Clicks a photo button inside your app to open camera
await $.platform.mobile.grantPermissionWhenInUse(); // Some devices require permission to be granted before taking a photo
await $.platform.mobile.takeCameraPhoto(); // Takes a photo using the camera
},
);
}
```
When you are using unsupported devices, you can provide your own selectors:
```dart
import 'package:example/main.dart';
import 'package:patrol/patrol.dart';
void main() {
patrolTest(
'Take a photo using android or iOS camera with custom selectors',
($) async {
await $.pumpWidgetAndSettle(const MyApp());
await $.tap(#addPhotoButton); // Clicks a photo button inside your app to open camera
await $.platform.mobile.grantPermissionWhenInUse(); // Some devices require permission to be granted before taking a photo
await $.platform.mobile.takeCameraPhoto(shutterButtonSelector: NativeSelector(
android: AndroidSelector(
resourceName: 'com.oplus.camera:id/shutter_button',
),
ios: IOSSelector(label: 'Take Picture'),
),
doneButtonSelector: NativeSelector(
android: AndroidSelector(
resourceName: 'com.oplus.camera:id/done_button',
),
ios: IOSSelector(label: 'Done'),
),
);
},
);
}
```
# Getting started
Try out Patrol's capabilities with few clicks! Follow this tutorial and get to know our framework.
## Try it out in Firebase Studio [#try-it-out-in-firebase-studio]
Create new project in Firebase Studio with Patrol demo project by clicking button below.
Check **`Mobile SDK Support (Flutter + Android Emulator)`** checkbox when creating a project.
Click **Import**.
Android emulator will open as preview and a test will start in develop mode. It takes a while, you can see logs in terminal
(it's called `onStart`, may be hidden by default). After test is finished, you can change test's or app's code and type `r`
in terminal to rerun it.
# Patrol Services
**Patrol is an open-source framework created and maintained by LeanCode under the Apache 2.0 License. It will always remain open source.**\
However, if your company wants to scale fast and accelerate Patrol’s adoption, we offer a set of value-added services on top of the core framework.\
Reach out to our team if you'd like support with setting up Patrol in your specific environment or if you're looking for experienced engineers to automate end-to-end testing for your product.\
As the creators of Patrol, we can provide the most accurate guidance on how to achieve your goals efficiently and with confidence.
# Our experience in Patrol Setup [#our-experience-in-patrol-setup]
Our experience comes from the implementation of Patrol in various environments
# Custom Patrol Setup for your Project [#custom-patrol-setup-for-your-project]
## Patrol Setup in Your Project (iOS & Android) [#patrol-setup-in-your-project-ios--android]
We support your team with a full Patrol implementation, including:
* adding Patrol to both iOS and Android projects
* adding a Patrol test for one test scenario inside your codebase
* ensuring the test runs locally on Android and iOS simulators/emulators
## CI/CD Workflow Setup (Azure DevOps, GitHub Actions, Bitrise) [#cicd-workflow-setup-azure-devops-github-actions-bitrise]
We configure end-to-end automation for Patrol tests, including:
* building Patrol-enabled app versions for test execution
* running tests on Firebase Test Lab for both iOS and Android virtual devices
* scheduling nightly test runs
* exporting logs as build artifacts
* generating and displaying test reports.
## Costs and pricing [#costs-and-pricing]
Patrol as a tool is free of charge and open-sourced. For the companies that want to speed up the adoption of Patrol and want to quickly have their first tests up and running, we recommend using our additional services related to Patrol Setup and Building Patrol Tests. Prices for this service depend on the scale of the project (startups/enterprise) and the number of tests that should be covered during initial setup.
| Offer for Startups / Small projects | Price |
| :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | :-------------------- |
| 1. Patrol Setup in your project (iOS + Android).
2. CI/CD workflow setup, assuming there is a CI/CD like Codemagic and a Firebase project for a given platform (iOS and/or Android).
3. The setup and production-ready, fully automated test with Patrol. | 3500 USD |
| Test(s) using Patrol:
- One test covers one user flow in the app, for example, onboarding or completing a transaction.
- Test scenarios need to be approved prior to the Patrol setup. | from 600 USD per test |
# Patrol Setup - pricing for Enterprises [#patrol-setup---pricing-for-enterprises]
| Offer for Enterprises / Large-scale projects | Price |
| :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | :---------------------------------------------------------------------------------------- |
| Patrol setup using custom CI/CD flows (possibly including Azure DevOps, GitHub Actions, Codemagic, etc.). | [Contact us](https://leancode.co/get-estimate?utm_source=patrol_page\&utm_medium=pricing) |
| Fully automated tests with Patrol:
- One test covers one user flow in the app, for example, onboarding or completing a transaction.
- Test scenarios need to be approved prior to the Patrol setup. | [Contact us](https://leancode.co/get-estimate?utm_source=patrol_page\&utm_medium=pricing) |
Follow [this link](https://leancode.co/products/patrol-setup-training?utm_source=patrol_page\&utm_medium=pricing) for the full offer.
# Patrol Consultation [#patrol-consultation]
If your team is considering adopting Patrol for your end-to-end UI testing, you can start with our Patrol Initial Consultation.\
We offer the first introductory meeting free of charge to discuss your challenges, clarify your testing goals, and outline the best approach for implementing Patrol in your environment. Follow [this link](https://meetings-eu1.hubspot.com/mateusz-wojtczak?uuid=c6bf91b7-58f8-4e33-9641-514f21df5e86) to schedule the introductory call.
# Patrol Training Program [#patrol-training-program]
We offer a comprehensive training program designed to upskill your team and help them become fully proficient in building and executing Patrol tests.\
Our training includes both lecture-style sessions and hands-on workshops, ensuring your team gains practical, real-world experience in using Patrol to test your Flutter applications.
Here is the detailed schedule of the training program:
| Type of training | Session type | Title of the session | Timing |
| :------------------ | :---------------------------- | :---------------------------------------------------------- | :---------------- |
| Core Curriculum | Lecture - Intro | Intro to Flutter & Patrol | 2h Lecture + Q\&A |
| Core Curriculum | Workshop - Using Patrol | Patrol Quiz App Workshop (+ DevTools Extension & Debugging) | 3h workshop |
| Core Curriculum | Lecture - Native automation | Patrol Native automation | 2h Lecture + Q\&A |
| Core Curriculum | Workshop - Setup | Patrol Setup (Android/iOS, Flavors, Dart-define variables) | 3h workshop |
| Advanced Curriculum | Lecture - CI/CD | CI/CD, Device Farms | 2h Lecture + Q\&A |
| Advanced Curriculum | Workshop - Advanced Scenarios | Sign in with Google, Location tracking | 3h workshop |
| Advanced Curriculum | Lecture - Architecture | Best practices, key structure, test structure | 2h Lecture + Q\&A |
Follow [this link](https://leancode.co/products/patrol-setup-training?utm_source=patrol_page\&utm_medium=pricing) to learn more about our Patrol Training Program.
# Overview
This section of the documentation is focused on running Patrol tests as part of
your Continuous Integration workflows.
Having tests doesn't bring you any benefits if you don't automatically verify
that they pass. We know this too well, and we're putting a lot of work into
making it easy to do so.
# Platforms
In this document, we'll outline a few ways to run Patrol UI tests of Flutter
apps.
Generally, the solutions for running UI tests of mobile apps can be divided into
2 groups:
* Device labs - platforms that provide access to mobile devices in the cloud. You
upload an app binary with tests to the device lab, which runs the tests and
reports the results back to you.
* Traditional – containers or VMs, either managed or self-hosted. In this
approach, you get access to the shell, so everything is possible. You manually
script what you want to achieve, which is usually: installing the Android/iOS
SDK tools, creating a virtual device, running tests, and collecting results.
There are quite a few solutions in each of these groups, and each is unique, but
generally, **device labs trade flexibility for ease of use**. They're a good fit
for most apps but make certain more complicated scenarios impossible.
# Device labs [#device-labs]
### Firebase Test Lab [#firebase-test-lab]
[Firebase Test Lab] is one of the most popular device labs. It is a good choice
for most projects.
You upload the app main app, the test app, select devices to run on, and after a
while, test results along with a video recording are available.
Firebase Test Lab has a large pool of physical and virtual devices.
See also:
* [Firebase Test Lab pricing]
### emulator.wtf [#emulatorwtf]
[emulator.wtf] is a fairly new solution created by Madis Pink and Tauno Talimaa. It
claims to provide a 2-5x speedup compared to Firebase Test Lab, and 4-20x
speedup compared to spawning emulators on CI machines. It works similarly to
Firebase Test Lab - you upload your main apk, test apk, select emulators to run
on, and the rest is up to emulator.wtf - it runs the tests and outputs results.
The emulators are indeed rock stable. Emulator.wtf automatically records videos
from test runs, and it presents the test results nicely.
It's a solid choice if you can accept that your tests will run only on Android
emulator.
Reports are available in JUnit.
See also:
* [emulator.wtf pricing]
### Xcode Cloud [#xcode-cloud]
[Xcode Cloud] is a CI/CD platform built into Xcode and designed expressly for
Apple developers. It doesn't support testing on Android.
Since integration tests written with Patrol are also native `XCTest`s, it should
be possible to run Patrol on Xcode Cloud. We plan to research it soon and share
our findings here.
### Other [#other]
Another popular device lab is [AWS Device Farm].
If your use-case is highly specific, you might want to build an in-house device
farm. A project that helps with this is [Simple Test Farm].
### Limitations [#limitations]
We mentioned above that device labs make certain scenarios impossible to
accomplish.
An example of such a scenario scanning a QR code. One of the apps we worked on had
this feature, and we wanted to test it because it was a critical part of the user
flow. When you have access to the shell filesystem (which you do have in the
"manual" approach, and don't have in the "device lab" approach), you can easily
[replace the scene that is visible in the camera's viewfinder][so_viewfinder].
This is not possible on device labs.
# Traditional [#traditional]
### Codemagic [#codemagic]
[Codemagic] is a popular CI/CD platform that integrates with Azure DevOps, GitHub, GitLab, Bitbucket, and
other self-hosted or cloud-based Git repositories.
It's also possible to run integration tests on Android directly on a Codemagic machine.
Here's a blog post about it: [Running Android integration tests on Codemagic].
However, this is generally not the recommended way to run your patrol tests. We recommend using device farms like [firebase test lab], [emulator.wtf] or others.
Codemagic will be great for preparing .apk files that you can upload to the device farms. To see documentation about using patrol in Codemagic workflows, please visit [codemagic/patrol documentation].
The full app example with all files is available in [codemagic/patrol-example-repository].
### GitHub Actions [#github-actions]
[GitHub Actions] is a very popular CI/CD platform, especially among open-source
projects thanks to unlimited minutes.
Unfortunately, running Flutter integration tests on GitHub Actions is not a
pleasant experience.
**Android**
We used the [ReactiveCircus/android-emulator-runner] GitHub Action to run
Android emulator on GitHub Actions. Our takeaway is this: Running an Android
emulator on the default GitHub Actions runner is a bad idea. It is slow to start and
unstable (apps crash randomly) and very slow. Really, really slow. We tried to
mitigate its instability by using [Test Butler], but it comes with its own
restrictions, most notably, it doesn't allow for Google Play Services.
**iOS**
We use the [futureware-tech/simulator-action] GitHub Action to run iOS simulator
on GitHub Actions is stable. But given that the iOS simulator is just that – a
simulator, not an emulator – the range of cases it can be used for is reduced.
For example, there's no easy way to disable an internet connection, which makes it
very hard to test the behavior of an app when offline.
Bear in mind that to run an iOS simulator on GitHub Actions, you have to use a
macOS runner. 1 minute on macos-latest counts as 10 minutes on ubuntu-latest.
You can also use a custom runner – more on that below.
Custom Runners Workflows on GitHub Actions can run on external runners, in
addition to default runners such as ubuntu-latest and macos-latest.
One example of such a custom runner provider is BuildJet. We tried running
Android emulator on it, hoping that the performance benefits it brings would
help with the abysmal stability, but we've found that, even though the emulator
works faster and is more stable, it sometimes just crashes with no actionable
error message.
### Other [#other-1]
There are many more CI/CD platforms. Some of the most popular include
[CircleCI], [CirrusCI], and [GitLab CI/CD]. There are also CI providers that are
focused specifically on mobile apps, for example [Bitrise] and [Codemagic]. If
you used these platforms, we (and other Patrol users) will be happy to hear
about your experiences!
[github actions]: https://github.com/features/actions
[aws device farm]: https://aws.amazon.com/device-farm
[emulator.wtf]: https://emulator.wtf
[emulator.wtf pricing]: https://emulator.wtf/pricing
[firebase test lab]: https://firebase.google.com/docs/test-lab
[firebase test lab pricing]: https://firebase.google.com/docs/test-lab/usage-quotas-pricing
[xcode cloud]: https://developer.apple.com/xcode-cloud
[test butler]: https://github.com/linkedin/test-butler
[reactivecircus/android-emulator-runner]: https://github.com/ReactiveCircus/android-emulator-runner
[futureware-tech/simulator-action]: https://github.com/futureware-tech/simulator-action
[simple test farm]: https://github.com/DeviceFarmer/stf
[so_viewfinder]: https://stackoverflow.com/questions/13818389/android-emulator-camera-custom-image
[circleci]: https://circleci.com
[cirrusci]: https://cirrus-ci.org
[gitlab ci/cd]: https://docs.gitlab.com/ee/ci
[bitrise]: https://bitrise.io
[codemagic]: https://codemagic.io/start
[codemagic/patrol documentation]: https://docs.codemagic.io/integrations/patrol-integration/
[codemagic/patrol-example-repository]: https://github.com/codemagic-ci-cd/codemagic-sample-projects/tree/main/integrations/patrol-demo-project
[running android integration tests on codemagic]: https://blog.codemagic.io/how-to-test-native-features-in-flutter-apps-with-patrol-and-codemagic/
# Advanced
We aim to make Patrol as simple as possible, but there are still a few matters
that we feel require some more attention. We'll explain them in this section.
### How is Patrol's `tap()` different from Flutter's `tap()`? [#how-is-patrols-tap-different-from-flutters-tap]
Let's consider this test, written without Patrol:
```dart
await tester.tap(find.byKey(Key('addComment')).first);
await tester.pumpAndSettle();
```
This code:
1. Immediately attempts to find the first widget with the `addComment` key
2. After finding the widget, it immediately attempts to tap on it
This is the default behavior, but in our experience, it's often a source of
flakiness. For example, the widget having `addComment` key might not be visible
at the time when the finder is run. This usually doesn't means that the test
should fail. Probably an HTTP request was made to fetch the post, and when the
fetching is done, the widget having `addComment` key will show up.
To achieve this behavior, you'd have to do:
```dart
while (find.byKey(Key('addComment')).first.evaluate().isEmpty) {
await tester.pump(Duration(milliseconds: 100));
}
await tester.tap(find.byKey(Key('addComment')).first);
await tester.pumpAndSettle();
```
Our tiny example got really big, but it's still got two problems.
1. If something goes wrong and `addComment` never shows up, we'll keep waiting
indefinitely.
2. The widget with `addComment` key might be present in the widget tree, but
still not be visible to the user. By default, Flutter's default
`WidgetTester` doesn't care. This is almost never desirable.
Fortunately, you don't have to overcome these problems. Patrol already did it!
Below is the same test, with all the above problems fixed, written with Patrol's
custom finders:
```dart
await $(#addComment).tap();
```
This code:
1. Attempts to find the first widget with `addComment` that is visible on
screen. If it's not found immediately, it keeps trying until it finds it, or
throws an exception if timeout.
2. Taps on it.
The timeout can be configured globally:
```dart
patrolWidgetTest(
'logs in successfully',
config: PatrolTestConfig(findTimeout: Duration(seconds: 10)),
($) async {
// your test code
},
);
```
You can also change the timeout ad-hoc:
```dart
await $(#addComment).tap(findTimeout: Duration(seconds: 30));
```
### You gotta pump it up! But which one to use? [#you-gotta-pump-it-up-but-which-one-to-use]
In Flutter, "pumping" means rendering frames to the screen.
If there are no frames to pump, no animations are pending, which usually means
that the next action during the test can be executed. It is an equivalent of
what a human tester would do while testing an app - they would wait until the
app's state stabilizes after they've done something. For example, they tap on a
button and get redirected to another screen, but the data that will be shown
there hasn't been loaded yet. In such a case, a human tester waits until a
loader (or other animation) finishes. Pumping mechanism does exactly this - it
renders consecutive frames on the screen. For how long, exactly? Usually, we
want to pump frames as long as they come. That's what [pumpAndSettle()] does.
`pumpAndSettle()` method is called by default inside all actions that can be
performed while testing - tapping, scrolling, entering text, and so on. You can
change that by setting the `settlePolicy` argument:
```dart
await $('Delete account').tap(settlePolicy: SettlePolicy.settle);
await $('Confirm').tap(settlePolicy: SettlePolicy.pump);
```
`SettlePolicy` is an `enum` with 3 values. The default is `SettlePolicy.settle`
but you can change it to `pump` or `trySettle`. Those values map to methods like this:
* `noSettle` -> `pump()`,
* `trySettle` -> `pumpAndTrySettle()`,
* `settle` -> `pumpAndSettle()`.
While `settle` and `pump` simply refer to Flutter's built-in methods,
`trySettle` is available only in Patrol. How is it different from other ones?
`pumpAndTrySettle()` is pretty much like `pumpAndSettle()`, the only difference
is that `pumpAndSettle()` throws an exception, if there were still new frames to
render after sonme defined timeout, while `pumpAndTrySettle()` does not. That's
why it has "`try`" in it's name.
When to use this new pumping method? Let us picture a scenario, in which we have
to deal with some animations. Let's say, that your app has some endless
animations, e.g. on a homescreen, to keep user's attention. You'd like to wait
for some things to happen, but using `pumpAndSettle`, you'll keep getting an
exception, because after some time, defined by `timeout`, there will be still
new frames to render. On the other hand, you still want to pump frames for some
time - if you didn't, the screen you want to interact with might not be rendered
yet, or it would have some widgets missing or data not yet loaded.
So, we decided to add a way to try settle - pump frames for some time (10
seconds by default), but if after that time there is still something new to
render - do nothing and continue the test.
We recommend using `pumpAndTrySettle()`, because it works with both kinds of
animations - finite and infinite. This settle policy will be new default in
future Patrol releases.
### How does `scrollTo()` work? [#how-does-scrollto-work]
The `scrollTo()` method is simple to use, yet very powerful. Here's how you use
it to scroll to and tap on the first widget with the `"Delete account"` text:
```dart
await $('Delete account').scrollTo().tap();
```
And here's how `scrollTo()` works:
1. Waits for at least 1 [Scrollable] widget (or whatever you provided in
`view` argument) to become visible
2. Scrolls this widget in its scrolling direction until the target
widget becomes visible
3. If the target widget becomes visible within timeout, it finishes, otherwise
it throws an exception
Most of the time, you use `scrollTo()` and it just works, but there's 1
important thing to keep in mind when using `scrollTo()`:
**`scrollTo()`, by default, scrolls the first `Scrollable` widget**
This default is reasonable and what you want most of the time. Unfortunately,
this behavior can sometimes cause problems in more complicated UIs, where more
than a single `Scrollable` widget is visible at the same time. In such cases we
strongly recommend explicitly specifying the `view` that `scrollTo()`
should scroll, to avoid the problem of the target widget never becoming visible
because the wrong widget was scrolled.
To demonstrate this problem, let's consider this very simple app:
```dart
class App extends StatelessWidget {
App({super.key});
@override
Widget build(BuildContext context) {
return MaterialApp(
home: Scaffold(
body: Column(
children: [
Expanded(child: ListView(key: Key('listView1'))),
Expanded(
child: ListView.builder(
key: Key('listView2'),
itemCount: 101,
itemBuilder: (context, index) => Text('index: $index'),
),
),
],
),
),
);
}
}
```
Now let's say that you're writing a test and want to scroll to and tap on the
first widget with the `"index: 100"` text (that is the last `Text` widget built
by the second `ListView` widget):
There's a high chance that you'd write this:
```dart
await $('index: 100').scrollTo().tap();
```
Unfortunately, running this test gives the `pumpAndSettle timed out` error.
That's because the `scrollTo()` was trying to scroll the first visible
`Scrollable` widget, which happens to be the first `ListView` (the one with
`listView1` key and no children).
To fix this problem, you have to explicitly specify which `Scrollable` you want
to use:
```dart
await $('index: 100').scrollTo(view: $(#listView2).$(Scrollable)).tap();
```
The above snippet will scroll the second `Scrollable` and find the widget with
`"index: 100"` text.
**Why so verbose?**
You might be wondering why `scrollTo(view: $(#listView2))` is not enough?
Why is it needed to look for a `Scrollable` widget inside the widget with the
`listView2` key?
This is because the [ListView] widget doesn't extend [Scrollable] – instead, it
builds a subclass of [Scrollable] itself. [This is a known Flutter
problem](https://github.com/flutter/flutter/issues/88762).
[listview]: https://api.flutter.dev/flutter/widgets/ListView-class.html
[scrollable]: https://api.flutter.dev/flutter/widgets/Scrollable-class.html
[pumpAndSettle()]: https://api.flutter.dev/flutter/flutter_test/WidgetTester/pumpAndSettle.html
# Using Patrol finders in widget tests
Since `patrol_finders` is a separate package, referenced in `patrol` package,
you can use it in your widget or golden tests, without depending on `patrol`.
Below you can find a short tutorial on how to use `$` in already existing widget tests.
### Install [#install]
First, add the `patrol_finders` package as a `dev_dependency` in your app's
`pubspec.yaml`. You can do this by executing the following command in the app's
directory:
```console
flutter pub add patrol_finders --dev
```
### Use [#use]
Our custom finders depend only on Flutter itself. This means you can use them
no matter if you're writing a mobile, web, desktop, or embedded app!
To use Patrol finders in your test files, first import it:
```dart
import 'package:patrol_finders/patrol_finders.dart';
```
Once imported, you can use it in widget tests:
```dart title="test/widget_test_with_patrol_finders.dart"
void main() {
testWidgets(
'counter is incremented when plus button is tapped',
(WidgetTester tester) async {
PatrolTester $ = PatrolTester(
tester: tester,
config: PatrolTesterConfig(),
);
await $.pumpWidget(const MyApp());
expect($('0'), findsOneWidget);
expect($('-1'), findsNothing);
await $(Icons.remove).tap();
expect($('0'), findsNothing);
expect($('-1'), findsOneWidget);
},
);
}
```
Or you can use our wrapper on `testWidgets` method, which initializes
`PatrolTester` object for you.
```dart title="test/patrol_widget_test.dart"
void main() {
patrolWidgetTest(
'counter is incremented when plus button is tapped',
(PatrolTester $) async {
await $.pumpWidget(const MyApp());
expect($('0'), findsOneWidget);
expect($('-1'), findsNothing);
await $(Icons.remove).tap();
expect($('0'), findsNothing);
expect($('-1'), findsOneWidget);
},
);
}
```
To run the test, simply execute:
```console
flutter test
```
# Overview
Flutter's finders are powerful, but not very intuitive to use.
We took them and made something awesome.
Thanks to Patrol's custom finders, you'll take your tests from this:
```dart
testWidgets('signs up', (WidgetTester tester) async {
await tester.pumpWidget(AwesomeApp());
await tester.pumpAndSettle();
await tester.enterText(
find.byKey(Key('emailTextField')),
'charlie@root.me',
);
await tester.pumpAndSettle();
await tester.enterText(
find.byKey(Key('nameTextField')),
'Charlie',
);
await tester.pumpAndSettle();
await tester.enterText(
find.byKey(Key('passwordTextField')),
'ny4ncat',
);
await tester.pumpAndSettle();
await tester.tap(find.byKey(Key('termsCheckbox')));
await tester.pumpAndSettle();
await tester.tap(find.byKey(Key('signUpButton')));
await tester.pumpAndSettle();
expect(find.text('Welcome, Charlie!'), findsOneWidget);
});
```
to this:
```dart
patrolWidgetTest('signs up', (PatrolTester $) async {
await $.pumpWidgetAndSettle(AwesomeApp());
await $(#emailTextField).enterText('charlie@root.me');
await $(#nameTextField).enterText('Charlie');
await $(#passwordTextField).enterText('ny4ncat');
await $(#termsCheckbox).tap();
await $(#signUpButton).tap();
await $('Welcome, Charlie!').waitUntilVisible();
});
```
# Usage
This page introduces Patrol finder system. Let's get our hands dirty
and find some widgets!
### Finding widgets [#finding-widgets]
Let's say you want to find some `Text` widget – nothing easier than that!
```dart
find.byType(Text);
```
Using Patrol finder, you'd write the above as:
```dart
$(Text);
```
Or let's find a `Text` widget with a specific text:
```dart
find.text('Subscribe');
```
Using Patrol finder, you'd write the above as:
```dart
$('Subscribe');
```
If you want to use semantics finders to locate widgets by their semantics properties, you can use flutter\_test finders inside Patrol finders.
```dart
await $(find.bySemanticsLabel('Edit profile')).tap();
```
Worth mentioning is also `Key`. The below lines are equivalent:
```dart
find.byKey(Key('loginButton'));
$(Key('loginButton'));
$(#loginButton);
```
For those wondering what is that `#` thing – it's a
[Symbol]! Yes, we're Dart
(ab)users.
All the types that can be passed to
`$`
are
[listed here][doc]
.
### Making assertions [#making-assertions]
Creating a finder doesn't do anything – it just *is*. Let's put them to use and
write a few simple assertions.
Here's how you can make sure that a widget with text `Log in` exists in the
widget tree:
```dart
expect(find.text('Log in'), findsOneWidget);
```
With our Patrol finders, you'd write the above as:
```dart
expect($('Log in'), findsOneWidget);
```
Alternatively, you could also use the `exists` getter, which returns true if the
finder finds at least 1 widget:
```dart
expect($('Log in').exists, equals(true));
```
We can also make sure that no widget exists, or that a particular number of
widgets exist:
```dart
expect(find.text("Can't touch this"), findsNothing);
expect(find.byType(Card), findsNWidgets(3));
```
The above expressed with Patrol finders:
```dart
expect($("Can't touch this"), findsNothing);
expect($(Card), findsNWidgets(3));
```
You could alternatively write the first line as:
```dart
expect($("Can't touch this").exists, equals(false));
```
[//]: # "not true, TODO: rewrite"
It's important to note that Flutter's default finder functions, such as
[findsNothing] and [findsOneWidget], check if the widget is present in the
widget tree, not if it is visible to the user, which is usually not what we're
interested in.
To check if the finder finds at least 1 *visible* widget, use the `visible`
getter:
```dart
expect($('Log in').visible, equals(true));
```
And to wait for at least 1 widget with the "Log in" text to become visible:
```dart
await $('Log in').waitUntilVisible();
```
### Performing actions [#performing-actions]
Finding widgets alone is cool, but what's even cooler is being able to tap on
them! Let's tap on the first "Subscribe" text:
```dart
await tester.tap(find.text('Subscribe').first);
```
It's usually a good practice to use `first`, because if there were multiple
"Subscribe" texts, `tap()` would throw an exception.
With Patrol, you get concise code, but you preserve the flexibility:
```dart
await $('Subscribe').tap();
```
What's very cool about Patrol's `tap()` is that it doesn't immediately fail if
the finder finds no visible widgets – instead, it waits for some time (which you
can specify globally in [PatrolTesterConfig] or as argument to the `tap()`
method) and taps on the first widget as soon as it becomes visible. This lets
you get rid of fixed timeouts and test your app just like a real user would.
If you wanted to tap on the third "Subscribe" text, you'd do:
```dart
await $('Subscribe').at(2).tap();
```
And if the "Subscribe" text was in a [Scrollable] widget, such as
[SingleChildScrollView] or [ListView], and you want to make sure that it is
visible (so you can `tap()` on it), you can scroll to it very easily:
```dart
await $('Subscribe').scrollTo().tap();
```
### Going deeper [#going-deeper]
But hey, these were very simple examples. In real apps, unfortunately, finding
widgets is not that easy.
Often, you'll need to tap on a widget which is in some other widget.
```dart
await tester.tap(
find.descendant(
of: find.byType(ListView),
matching: find.text('Subscribe'),
).first
);
```
Flutter's finders are starting to grow, while Patrol stays lean:
```dart
await $(ListView).$('Subscribe').tap();
```
Now, we also make sure that the `Subscribe` text is in a `ListTile`:
```dart
await tester.tap(
find.descendant(
of: find.byType(ListView),
matching: find.descendant(
of: find.byType(ListTile),
matching: find.text('Subscribe'),
),
).first
);
```
Hey, this is starting to look complex! Fortunately, you have Patrol:
```dart
await $(ListView).$(ListTile).$('Subscribe').tap();
```
Sometimes, you might want to perform a lookahead check. Let's say that you want
tap on the first widget with the `Key('learnMore')` that is a descendant of some
`ListTile`, but that `ListTile` must also have the `Text` descendant with the
`Activated` text.
If you were to express the above as a `Finder`, you'd get:
```dart
await tester.tap(
find.ancestor(
of: find.text('Activated'),
matching: find.descendant(
of: find.byType(ListTile),
matching: find.byKey(Key('learnMore')),
),
).first
);
```
With the help of Patrol's custom finders, it's much easier:
```dart
await $(ListTile).containing('Activated').$(#learnMore).tap();
```
Sometimes, however, the logic required to find a widget cannot be expressed by
the descendant/ancestor relationship like above. In situations like this,
when all ways of finding widgets known to you fail, Patrol has an ace up its
sleeve: the [which()] method. You can use it to find widgets by their
properties. A few examples include:
.
* entering a text into a text field with no text entered:
```dart
await $(#cityTextField)
.which((widget) => widget.controller.text.isNotEmpty)
.enterText('Warsaw, Poland');
```
* asserting that the icon has the correct color:
```dart
await $(Icons.error)
.which((widget) => widget.color == Colors.red)
.waitUntilVisible();
```
* asserting that the button is disabled and has the correct color
```dart
await $('Delete account')
.which((button) => !button.enabled)
.which(
(btn) => btn.style?.backgroundColor?.resolve({}) == Colors.red,
)
.waitUntilVisible();
```
### Falling back [#falling-back]
What's cool about Patrol is that it builds on top of `flutter_test` instead of
replacing it. This means that you can freely mix Patrol's finders with finders
from `flutter_test`, `PatrolTester` with `WidgetTester`, and so on.
Here's how you can access the default `WidgetTester`:
```dart
patrolWidgetTest('adds comment', (PatrolTester $) async {
final WidgetTester tester = $.tester;
await tester.enterText(find.byKey(Key('commentTextField')), 'Very nice!');
});
```
[doc]: https://pub.dev/documentation/patrol_finders/latest/patrol_finders/createFinder.html
[which()]: https://pub.dev/documentation/patrol_finders/latest/patrol_finders/PatrolFinder/which.html
[findsNothing]: https://api.flutter.dev/flutter/flutter_test/findsNothing-constant.html
[findsOneWidget]: https://api.flutter.dev/flutter/flutter_test/findsOneWidget-constant.html
[PatrolTesterConfig]: https://pub.dev/documentation/patrol_finders/latest/patrol_finders/PatrolTesterConfig-class.html
[singlechildscrollview]: https://api.flutter.dev/flutter/widgets/SingleChildScrollView-class.html
[listview]: https://api.flutter.dev/flutter/widgets/ListView-class.html
[Scrollable]: https://api.flutter.dev/flutter/widgets/Scrollable-class.html
[Symbol]: https://api.dart.dev/dart-core/Symbol-class.html
# Allure
## Overview [#overview]
If you're using [Allure] to report your test results, you can use the
alternative test runner to get more detailed test report.
We decided not to package this alternative runner together with Patrol because
it'd make Patrol depend on Allure, which is not desirable. Instead, you can
easily do it yourself.
This guide assumes basic familiarity with Allure. To get started, see:
* [official Allure documentation]
* [allure-framework/allure2 repository]
This integration is currently Android-only.
Before you proceed with the steps listed below, make sure that you've
completed the [native setup] guide.
### Add dependencies and change runner [#add-dependencies-and-change-runner]
First, you have to modify the **app-level build.gradle**:
```groovy title="android/app/build.gradle"
android {
// ...
defaultConfig {
// ...
// Replace the existing "testInstrumentationRunner" line with:
testInstrumentationRunner "pl.leancode.patrol.example.AllurePatrolJUnitRunner"
}
// ...
}
dependencies {
androidTestImplementation "io.qameta.allure:allure-kotlin-model:2.4.0"
androidTestImplementation "io.qameta.allure:allure-kotlin-commons:2.4.0"
androidTestImplementation "io.qameta.allure:allure-kotlin-junit4:2.4.0"
androidTestImplementation "io.qameta.allure:allure-kotlin-android:2.4.0"
}
```
Replace `pl.leancode.patrol.example` with your app's package name.
See also:
* [the README of allure-kotlin library][allure_kotlin]
### Create alternative runner [#create-alternative-runner]
Create a new Kotlin file in the same directory as **MainActivityTest.java** and
paste the following code, replacing the package:
```kotlin title="AllurePatrolJUnitRunner.kt"
package pl.leancode.patrol.example // replace "pl.leancode.patrol.example" with your app's package
import android.os.Bundle
import io.qameta.allure.android.AllureAndroidLifecycle
import io.qameta.allure.android.listeners.ExternalStoragePermissionsListener
import io.qameta.allure.android.writer.TestStorageResultsWriter
import io.qameta.allure.kotlin.Allure
import io.qameta.allure.kotlin.junit4.AllureJunit4
import io.qameta.allure.kotlin.util.PropertiesUtils
import pl.leancode.patrol.PatrolJUnitRunner
class AllurePatrolJUnitRunner : PatrolJUnitRunner() {
override fun onCreate(arguments: Bundle) {
Allure.lifecycle = createAllureAndroidLifecycle()
val listenerArg = listOfNotNull(
arguments.getCharSequence("listener"),
AllureJunit4::class.java.name,
ExternalStoragePermissionsListener::class.java.name.takeIf { useTestStorage }
).joinToString(separator = ",")
arguments.putCharSequence("listener", listenerArg)
super.onCreate(arguments)
}
private fun createAllureAndroidLifecycle() : AllureAndroidLifecycle {
return createDefaultAllureAndroidLifecycle()
}
private fun createDefaultAllureAndroidLifecycle() : AllureAndroidLifecycle {
if (useTestStorage) {
return AllureAndroidLifecycle(TestStorageResultsWriter())
}
return AllureAndroidLifecycle()
}
private val useTestStorage: Boolean
get() = PropertiesUtils.loadAllureProperties()
.getProperty("allure.results.useTestStorage", "true")
.toBoolean()
}
```
In the snippet above, remember to replace the `package
pl.leancode.patrol.example` line at the top of the file with your app's
package name!
### Create allure.properties [#create-allureproperties]
This is required if you enabled the `clearPackageData` option for Android Test
Orchestrator. If you enabled that option but don't create the
`allure.properties` file as below, your tests reports will be cleared after each
test.
```txt title="android/app/src/main/res/allure.properties"
allure.results.useTestStorage=true
```
### Add rules to MainActivityTest [#add-rules-to-mainactivitytest]
Finally, modify the **MainActivityTest.java**. You'll add a 3 rules, which add
the following features:
* automatically take a screenshot at the end of each test
* automatically dump the window hierarchy at the end of each test
* automatically embed the logcat into the report
You can simply copy-paste the following code (remember to replace the package
name):
```java title="MainActivityTest.java"
package pl.leancode.patrol.example; // replace "pl.leancode.patrol.example" with your app's package
import androidx.test.platform.app.InstrumentationRegistry;
import org.junit.Rule;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.junit.runners.Parameterized;
import org.junit.runners.Parameterized.Parameters;
import io.qameta.allure.android.rules.LogcatRule;
import io.qameta.allure.android.rules.ScreenshotRule;
import io.qameta.allure.android.rules.WindowHierarchyRule;
import pl.leancode.patrol.PatrolJUnitRunner;
@RunWith(Parameterized.class)
public class MainActivityTest {
@Rule
public ScreenshotRule screenshotRule = new ScreenshotRule(ScreenshotRule.Mode.END, "ss_end");
@Rule
public WindowHierarchyRule windowHierarchyRule = new WindowHierarchyRule();
@Rule
public LogcatRule logcatRule = new LogcatRule();
@Parameters(name = "{0}")
public static Object[] testCases() {
PatrolJUnitRunner instrumentation = (PatrolJUnitRunner) InstrumentationRegistry.getInstrumentation();
instrumentation.setUp(MainActivity.class);
instrumentation.waitForPatrolAppService();
return instrumentation.listDartTests();
}
public MainActivityTest(String dartTestName) {
this.dartTestName = dartTestName;
}
private final String dartTestName;
@Test
public void runDartTest() {
PatrolJUnitRunner instrumentation = (PatrolJUnitRunner) InstrumentationRegistry.getInstrumentation();
instrumentation.runDartTest(dartTestName);
}
}
```
In the snippet above, remember to replace the `package
pl.leancode.patrol.example` line at the top of the file with your app's
package name!
### Retrieve the report [#retrieve-the-report]
Run the tests with `patrol test` as usual.
After the tests are complete, create a directory for them, for example:
```bash
mkdir -p ./build/reports
```
and then retrieve the results from the device:
```bash
adb exec-out sh -c 'cd /sdcard/googletest/test_outputfiles && tar cf - allure-results' | tar xvf - -C build/reports
```
Finally, serve the results with Allure:
```bash
allure serve ./build/reports/allure-results
```
If you're using Homebrew, `brew install allure` is the quickest way to get
Allure.
[native setup]: /documentation
[allure]: https://qameta.io/allure-report
[allure_kotlin]: https://github.com/allure-framework/allure-kotlin/blob/master/README.md
[official Allure documentation]: https://docs.qameta.io/allure-report
[allure-framework/allure2 repository]: https://github.com/allure-framework/allure2
# BrowserStack
## Setup [#setup]
[BrowserStack App Automate] is a popular cloud device farm. You can use it to run your tests on real devices.
### Change runner [#change-runner]
Modify the **app-level build.gradle.kts**:
```groovy title="android/app/build.gradle.kts"
android {
// ...
defaultConfig {
//...
testInstrumentationRunner = "pl.leancode.patrol.BrowserstackPatrolJUnitRunner"
}
// ...
}
// ...
```
That's it! You can now use `bs_android` to schedule a test run.
You need to do a [Setup for physical iOS devices] first.
We need to convert your tests to use [Xcode test plans].
Make sure that the project name is "Runner" and the scheme is named "Runner" - this is the default name for the Flutter project.
Open your project in Xcode and edit the scheme:
Go to the **Test** tab and convert your tests to use test plans:
Create from scheme:
Rename to "TestPlan" and save:
It has to be named "TestPlan" to work with the `bs_ios` script.
Now, you can schedule a test run using the `bs_ios` script.
You can choose between running tests in a recommended way using scripts or manually:
## Schedule tests using scripts [#schedule-tests-using-scripts]
We recommend using the [bs\_android][bs_android] and [bs\_ios][bs_ios] scripts to schedule test runs.
They are part of LeanCode's [mobile-tools]. If you're using Homebrew, you can install it with:
```bash
brew tap leancodepl/tools
brew install mobile-tools
```
The scripts require the `BS_CREDENTIALS` environment variable
to be set so it can authenticate with BrowserStack:
```bash
export BS_CREDENTIALS="YOUR_USERNAME:YOUR_ACCESS_KEY"
```
Get your username and access on [BrowserStack's account page][bs_account].
Now reload your shell (e.g. `exec zsh`)
### Usage [#usage]
The scripts forward all its options and flags to `patrol build`, so you can use it like this:
```bash
bs_android \
--target patrol_test/example_test.dart,patrol_test/another_test.dart \
--verbose \
--dart-define 'KEY_EXAMPLE=VALUE_EXAMPLE'
```
Full example:
```
$ export BS_PROJECT=AwesomeApp # optional
$ export BS_ANDROID_DEVICES="[\"Google Pixel 4-10.0\"]" # optional
$ bs_android
• Building apk with entrypoint test_bundle.dart...
✓ Completed building apk with entrypoint test_bundle.dart (11.0s)
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 87.4M 100 235 100 87.4M 7 2857k 0:00:33 0:00:31 0:00:02 2052k
Uploaded app, url: bs://fb61a714e1a0c60e2578d940dad52b74da244d54
Uploaded test, url: bs://a715b1231d41ac627bd683f1b16c28476babd72e
{"message":"Success","build_id":"a30440db559fcab65554ab0273437f3bd45d761b"}
Scheduled test execution
```
That's all! "Success" means that the test execution was scheduled successfully.
You can follow BrowserStack's docs and/or follow the code of `bs_android` and `bs_ios` scripts:
1. Build the app under test and the instrumentation app ([see docs][patrol build])
2. Upload the app under test APK to BrowserStack ([see Android docs][bs_android_app_docs]) ([see iOS docs][bs_ios_app_docs])
3. Upload the instrumentation app APK to BrowserStack ([see Android docs][bs_android_test_docs]) ([see iOS docs][bs_ios_test_docs])
4. Start test execution on BrowserStack ([see Android docs][bs_execute_android_docs]) ([see iOS docs][bs_execute_ios_docs])
After scheduling the test execution, you can check the status of the test execution in the [App Automate dashboard][bs_app_automate_dashboard].
If you need to change the test configuration, check out full list of available devices and OS versions in the [BrowserStack Browsers & Devices][bs_devices].
[BrowserStack App Automate]: https://www.browserstack.com/app-automate
[mobile-tools]: https://github.com/leancodepl/mobile-tools
[bs_account]: https://www.browserstack.com/accounts/profile
[bs_app_automate_dashboard]: https://app-automate.browserstack.com/dashboard/v2
[Setup for physical iOS devices]: /documentation/physical-ios-devices-setup
[Xcode test plans]: https://developer.apple.com/documentation/xcode/organizing-tests-to-improve-feedback
[bs_android]: https://github.com/leancodepl/mobile-tools/blob/master/bin/bs_android
[bs_ios]: https://github.com/leancodepl/mobile-tools/blob/master/bin/bs_ios
[bs_devices]: https://www.browserstack.com/list-of-browsers-and-platforms/app_automate
[patrol build]: /cli-commands/build
[bs_android_app_docs]: https://www.browserstack.com/docs/app-automate/api-reference/espresso/apps#upload-an-app
[bs_android_test_docs]: https://www.browserstack.com/docs/app-automate/api-reference/espresso/tests#upload-a-test-suite
[bs_execute_android_docs]: https://www.browserstack.com/docs/app-automate/api-reference/espresso/builds#execute-a-build
[bs_ios_app_docs]: https://www.browserstack.com/docs/app-automate/api-reference/xcuitest/apps#upload-an-app
[bs_ios_test_docs]: https://www.browserstack.com/docs/app-automate/api-reference/xcuitest/tests#upload-a-test-suite
[bs_execute_ios_docs]: https://www.browserstack.com/docs/app-automate/api-reference/xcuitest/builds#execute-a-build
# Firebase Test Lab
There are many device lab providers. Below we're showing how to run Patrol tests
on [Firebase Test Lab], because it's popular in the Flutter community, but the
instructions should be similar for other device farms, such as [AWS Device
Farm][aws_device_farm].
Before you proceed with the steps listed below, make sure that you've
completed the [native setup] guide.
To run the integration tests on Android, you need 2 apps: the app itself
(often called the "app under test") and the test intrumentation app.
To build these apps, run:
```
patrol build android --target patrol_test/example_test.dart
```
Once you have built the apks, use the [gcloud] tool to run them on Firebase
Test Lab:
```
gcloud firebase test android run \
--type instrumentation \
--use-orchestrator \
--app build/app/outputs/apk/debug/app-debug.apk \
--test build/app/outputs/apk/androidTest/debug/app-debug-androidTest.apk \
--timeout 1m \
--device model=MediumPhone.arm,version=34,locale=en,orientation=portrait \
--record-video \
--environment-variables clearPackageData=true
```
You must [install the gcloud tool] first. [Here][gcloud_android] you can learn
more about all available options and flags.
The environment variable `clearPackageData=true` tells orchestrator to clear the
package data between test runs. Keep in mind that it clears only the data of your
app, not other data on the device, e.g. Chrome.
It's convenient to create a shell script to avoid typing that long command
every time. You might want to take a look at Patrol example app's
[run\_android\_testlab script][example_android_script].
On Android, all permissions are granted by default. This behavior can be
changed using the [alpha version of the gcloud tool].
To run the integration tests on iOS, you need 2 apps: the app itself
(often called the "app under test") and the test intrumentation app.
First, build your Flutter app, choosing the integration test file as target:
For simulations:
```
patrol build ios --target patrol_test/example_test.dart --debug --simulator
```
For physical devices:
```
patrol build ios --target patrol_test/example_test.dart --release
```
`patrol build ios` outputs paths to the built app binaries, for example:
```
$ patrol build ios -t patrol_test/example_test.dart --release
• Building app with entrypoint example_test.dart for iOS device (release)...
✓ Completed building app with entrypoint example_test.dart for iOS device (31.5s)
build/ios_integ/Build/Products/Release-iphoneos/Runner.app (app under test)
build/ios_integ/Build/Products/Release-iphoneos/RunnerUITests-Runner.app (test instrumentation app)
build/ios_integ/Build/Products/Runner_iphoneos16.2-arm64.xctestrun (xctestrun file)
```
Firebase Test Lab requires these files to be packaged together in a zip
archive. To create the archive:
```
pushd build/ios_integ/Build/Products
zip -r ios_tests.zip Release-iphoneos Runner_iphoneos16.2-arm64.xctestrun
popd
```
Finally, upload the `ios_tests.zip` to Firebase Test Lab for execution:
```
gcloud firebase test ios run \
--test build/ios_integ/Build/Products/ios_tests.zip \
--device model=iphone8,version=16.2,locale=en_US,orientation=portrait
```
You must [install the gcloud tool] first. [Here][gcloud_ios] you can learn
more about all available options and flags.
If your `.xctestrun` file has different iOS version in its name than the
device you're running on, simply rename the `.xctestrun` so that the version
matches.
It's convenient to create a shell script to avoid typing that long command
every time. You might want to take a look at Patrol example app's
[run\_ios\_testlab script][example_ios_script].
[native setup]: /documentation
[gcloud]: https://cloud.google.com/sdk/gcloud
[example_android_script]: https://github.com/leancodepl/patrol/blob/master/dev/e2e_app/run_android_testlab
[example_ios_script]: https://github.com/leancodepl/patrol/blob/master/dev/e2e_app/run_ios_testlab
[firebase test lab]: https://firebase.google.com/products/test-lab
[aws_device_farm]: https://aws.amazon.com/device-farm
[install the gcloud tool]: https://cloud.google.com/sdk/docs/install
[gcloud_android]: https://cloud.google.com/sdk/gcloud/reference/firebase/test/android/run
[gcloud_ios]: https://cloud.google.com/sdk/gcloud/reference/firebase/test/ios/run
[alpha version of the gcloud tool]: https://cloud.google.com/sdk/gcloud/reference/alpha/firebase/test/android/run#--grant-permissions
# LambdaTest
# LambdaTest overview [#lambdatest-overview]
[LambdaTest App Test Automation] is a popular cloud device farm.
This integration is currently Android-only.
### Change runner [#change-runner]
Modify the **app-level build.gradle**:
```groovy title="android/app/build.gradle"
android {
// ...
defaultConfig {
//...
testInstrumentationRunner "pl.leancode.patrol.LambdaTestPatrolJUnitRunner"
}
// ...
}
// ...
```
### Upload to LambdaTest [#upload-to-lambdatest]
To run Android UI tests on LambdaTest:
1. Upload the app under test APK to LambdaTest ([see docs][LT_app_docs])
2. Upload the instrumentation app APK to LambdaTest ([see docs][LT_test_docs])
3. Start test execution on LambdaTest ([see docs][LT_execute_docs])
```
$ export LAMBDATEST_PROJECT=AwesomeApp # optional
$ export LAMBDATEST_DEVICES="[\"Pixel 7 Pro-13\"]" # optional
• Building apk with entrypoint test_bundle.dart...
✓ Completed building apk with entrypoint test_bundle.dart (11.0s)
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 82.4M 100 255 100 82.4M 7 2897k 0:00:31 0:00:36 0:00:02 2051k
Uploaded app, "app_id": "lt://APP1016047291733313441063634",
Uploaded testsuite, "app_id": "lt://APP1016047291733312896265135",
{
"status": [
"Success"
],
"buildId": [
"5875687"
],
"message": [
""
]
}
```
[LT_app_docs]: https://www.lambdatest.com/support/docs/getting-started-with-espresso-testing/#running-your-first-test-a-step-by-step-guide
[LT_test_docs]: https://www.lambdatest.com/support/docs/getting-started-with-espresso-testing/#step-2-upload-your-test-suite
[LT_execute_docs]: https://www.lambdatest.com/support/docs/getting-started-with-espresso-testing/#step-3-executing-the-test
[LambdaTest App Test Automation]: https://www.lambdatest.com/app-test-automation
# Agent skills
Patrol ships [agent skills](https://github.com/leancodepl/patrol/tree/master/skills) — focused,
on-demand instructions in the open [Agent Skills](https://agentskills.io/) format (`SKILL.md`) that
teach an AI coding agent *how* to write Patrol tests following best practices. They complement
[Patrol MCP](/documentation/other/patrol-mcp): the MCP gives the agent the tools to run tests, while
a skill teaches it how to use those tools to write tests well.
See the [skills catalog in the Patrol repo](https://github.com/leancodepl/patrol/tree/master/skills)
for the available skills and per-agent install and update instructions.
# Debugging Patrol tests
If you want to debug your application during patrol tests,
you can do in Visual Studio Code by attaching a debugger to the running process.
Here is how you can do it:
1. In your `launch.json` file, add a new configuration for attaching debugger to a process:
```json
{
"name": "attach debugger",
"request": "attach",
"type": "dart",
"cwd": "patrol_test",
"vmServiceUri": "${command:dart.promptForVmService}"
}
```
2. Run your patrol tests using `develop` command with the same arguments as you would normally do.
3. When the tests will start running, at some point you will see a message with a link to Patrol devtools extension.
Copy the last part of the URI from the message.
Eg. for this link:
`Patrol DevTools extension is available at http://127.0.0.1:9104/patrol_ext?uri=http://127.0.0.1:52263/F2-CH29gR1k=/`
copy `http://127.0.0.1:52263/F2-CH29gR1k=/`.
You'll see 2 similar logs. First one looks like this:
`The Dart VM service is listening on http://127.0.0.1:63725/57XmBI_pwSA=/`
Ignore it - this link is incorrect, wait for the one that says about devtools extension.
4. From "Run and Debug" tab in Visual Studio Code, select the configuration you have created in step 1.
You will be prompted to enter the VM service URI. Paste the URI you copied in step 3.
5. Once the debugger is attached, you can set breakpoints and debug your application as you would normally do.
Intellij/Android Studio does not support attaching a debugger to a running process via Observatory Uri.
Therefore you cannot achieve the same behavior in those IDEs (See this [issue]).
[issue]: https://github.com/flutter/flutter-intellij/issues/2250
# Effective Patrol
Over the past months, we've written many Patrol tests and often learned the hard
way what works well and what doesn't. We're sharing our findings hoping that
they'll help you write robust tests.
This document follows
[RFC 2119][rfc2119]
.
### PREFER using keys to find widgets [#prefer-using-keys-to-find-widgets]
Patrol's custom finders are very powerful, and you might often be inclined to
find the widget you want in a variety of ways. While we're encouraging you to
explore and play with Patrol's custom finders, we are quite confident that keys
are the best way to find widgets.
**Why not strings?**
At first, strings might seem like a good way to find widgets.
They'll get increasingly annoying to work with as your app grows and changes,
for example, when the strings in your app change.
Using strings stops making any sense when you have more than 1 language in your
app. Using strings in such case is asking for trouble.
**Why not classes?**
There are 2 problems with using classes.
First is that they hurt your test's readability. You want to tap on *the* login
button or enter text into *the* username field. You don't want to tap on, say,
the third button and enter text into the second text field.
The second problem is that classes are almost always an implementation detail.
As a tester, you shouldn't care if something is a `TextButton` or an
`OutlineButton`. You care that it is *the* login button, and you want to tap on
it. In most cases, that login button should have a key.
Let's consider this simple example:
```dart
await $(LoginForm).$(Button).at(1).tap(); // taps on the second ("login") button
```
This works, but the code is not very self-explanatory. To make it understandable
at glance, you had to add a comment.
But if you assigned a key to the login button, the above could be simplified to:
```dart
await $(#loginButton).tap();
```
Much better!
Let's see another example:
```dart
await $(Select).tap(); // taps on the first Select
```
If the type parameter is changed from `String` to, for example, some specialized
`PersonData` model, that finder won't find anything. You'd have to update it to:
```dart
await $(Select).tap();
```
You had to change your test, even though nothing changed from the user's
perspective. This is usually a sign that you rely too much on classes to find
widgets.
This whole section could be summed up to the simple maxim:
> Have tester's mindset.
Treat your finders as if they were the tester's eyes.
### CONSIDER having a file where all keys are defined [#consider-having-a-file-where-all-keys-are-defined]
The number of keys will get bigger as your app grows and you write more tests.
To keep track of them, it's a good idea to keep all keys in, say,
`lib/keys.dart` file.
```dart title="lib/keys.dart"
import 'package:flutter/foundation.dart';
typedef K = Keys;
class Keys {
const Keys();
static const usernameTextField = Key('usernameTextField');
static const passwordTextField = Key('passwordTextField');
static const loginButton = Key('loginButton');
static const forgotPasswordButton = Key('forgotPasswordButton');
static const privacyPolicyLink = Key('privacyPolicyLink');
}
```
Then you can use it in your app's and tests' code:
```dart title="In app UI code"
@override
Widget build(BuildContext context) {
return Column(
children: [
/// some widgets
TextField(
key: K.usernameTextField,
// some other TextField properties
),
// more widgets
],
);
}
```
```dart title="In app test code"
void main() {
patrolTest('logs in', (PatrolIntegrationTester $) {
// some code
await $(K.usernameTextField).enterText('CoolGuy');
// more code
});
}
```
This is a good way to make sure that the same keys are used in app and tests. No
more typos!
### PREFER having one test path [#prefer-having-one-test-path]
Good tests test one feature, and test it well (this applies to all tests, not
only Patrol tests). This is often called the "main path". Try to introduce as
little condional logic as possible to help keep the main path straight. In
practice, this usually comes down to having as few `if`s as possible.
Keeping your test code simple and to the point will also help you in debugging
it.
### DO add a good test description explaining the test's purpose [#do-add-a-good-test-description-explaining-the-tests-purpose]
If your app is non-trivial, your Patrol test will become long pretty quickly.
You may be sure now that you'll always remember what the 200 line long test
you've just written does and are (rightfully) very proud of it.
Believe us, in 3 months you will not remember what your test does. This is why
the first argument to `patrolTest` is the test description. Use it well!
```dart
// GOOD
import 'package:awesome_app/main.dart';
import 'package:patrol/patrol.dart';
void main() {
patrolTest(
'signs up for the newsletter and receives a reward',
($) async {
await $.pumpWidgetAndSettle(AwesomeApp());
await $(#phoneNumber).enterText('800-555-0199');
await $(#loginButton).tap();
// more code
},
);
}
```
```dart
// BAD
void main() {
patrolTest(
'test',
($) async {
await $.pumpWidgetAndSettle(AwesomeApp());
await $(#phoneNumber).enterText('800-555-0199');
await $(#loginButton).tap();
// more code
},
);
}
```
[rfc2119]: https://www.ietf.org/rfc/rfc2119.txt
# Patrol DevTools Extension
A powerful Flutter DevTools extension that allows you to **inspect native UI elements** on Android and iOS devices while developing your Patrol tests. This extension provides a tree view of native UI components, making it easier to write accurate selectors for your integration tests.
## Features [#features]
* **Native UI Tree Inspection**: Browse the complete hierarchy of native UI elements on your device
* **Element Details**: View detailed information about each native element (bounds, text, accessibility properties, etc.)
* **Cross-platform Support**: Works with both Android and iOS applications
* **Live Updates**: Refresh the UI tree to see real-time changes
* **Test Integration**: Copy element selectors directly for use in your Patrol tests
## Quick Start [#quick-start]
Launch Your app with test in Development Mode
To use the DevTools extension, start your app in Patrol development mode:
```bash
patrol develop -t patrol_test/example_test.dart
```
This command will:
* Launch your test on the connected device/simulator in develop mode
* Start the Flutter DevTools server
* Print a clickable link to the DevTools interface in your terminal
Open the DevTools Extension
When `patrol develop -t` is running, look for output similar to this in your terminal:
```
Patrol DevTools extension is available at:
http://127.0.0.1:9102/patrol_ext?uri=http://127.0.0.1:58463/MOAGppLU9BU=/
```
**Click on this link** to open Flutter DevTools in your browser.
If logs are cluttering your terminal and making it hard to find the DevTools link, you can use the `--open-devtools` flag to automatically open DevTools:
> ```bash
> patrol develop -t patrol_test/example_test.dart --open-devtools
> ```
Navigate to the Patrol Extension
Once DevTools opens:
* By first time you need to Enable extension, just click the button that will shows up
* Look for the **"Patrol"** tab in the top navigation bar
* Click on it to open the Patrol DevTools Extension
Load the UI Tree
* Make sure that your wanted native view is visible on your device/simulator
* Click the **Refresh** button (🔄) in the Patrol extension
* You should see the native UI tree populate in the left panel
### Flutter DevTools [#flutter-devtools]
After opening the Patrol DevTools link, you can also use the Flutter Inspector to see widgets and their
properties in your Flutter app. To make it work, you need to add the path to your app's lib folder (e.g., user/my\_example\_app/lib) under settings (⚙️) on the DevTools page.
### DevTools Interface [#devtools-interface]
#### Tree View Controls [#tree-view-controls]
* **RAW button**: Shows native tree detailed information (You need to refresh native tree after toggle)
* **📱 Full node names**: Shows full node names
* **🔄 Refresh tree**: Load current UI tree
## Using Discovered Elements in Tests [#using-discovered-elements-in-tests]
When you find elements in the inspector, you can create cross-platform selectors that work on both Android and iOS:
```dart
// Cross-platform button using unique identifiers
await $.platform.mobile.tap(NativeSelector(
android: AndroidSelector(resourceId: 'com.example:id/login_btn'),
ios: IOSSelector(identifier: 'loginButton'),
));
// Using class name (Android) and element type (iOS)
await $.platform.mobile.tap(NativeSelector(
android: AndroidSelector(className: 'android.widget.Button'),
ios: IOSSelector(elementType: 'XCUIElementTypeButton'),
));
```
### Key Properties Reference [#key-properties-reference]
#### Android Properties [#android-properties]
| Property | Description | Example |
| -------------------- | ---------------------------------- | ----------------------- |
| `resourceName` | Unique resource ID (most reliable) | `com.app:id/login_btn` |
| `text` | Visible text content | `"Sign In"` |
| `className` | UI element type | `android.widget.Button` |
| `contentDescription` | Accessibility description | `"Login button"` |
| `applicationPackage` | App package name | `com.example.app` |
Full list of Android properties can be found:
[https://pub.dev/documentation/patrol/latest/patrol/AndroidSelector-class.html](https://pub.dev/documentation/patrol/latest/patrol/AndroidSelector-class.html)
#### iOS Properties [#ios-properties]
| Property | Description | Example |
| ------------- | --------------------------------- | ----------------------- |
| `identifier` | Unique identifier (most reliable) | `loginButton` |
| `elementType` | UI element type | `XCUIElementTypeButton` |
| `label` | Accessibility label | `"Sign In"` |
| `title` | Element title | `"Login"` |
Full list of iOS properties can be found:
[https://pub.dev/documentation/patrol/latest/patrol/IOSSelector-class.html](https://pub.dev/documentation/patrol/latest/patrol/IOSSelector-class.html)
# Patrol MCP
## Overview [#overview]
Patrol MCP is an MCP (Model Context Protocol) server that lets AI assistants
run and manage Patrol tests in Flutter projects. It wraps `patrol develop`
and exposes it as a set of MCP tools.
* [patrol\_mcp on pub.dev](https://pub.dev/packages/patrol_mcp)
* [Setup guide & IDE configuration (README)](https://github.com/leancodepl/patrol/blob/master/packages/patrol_mcp/README.md)
## Version Compatibility [#version-compatibility]
**Most users can ignore this table.** If you use `patrol_cli` installed
globally (via `dart pub global activate patrol_cli`), the version pinning
does not affect you — the global CLI is a separate binary and `patrol_mcp`
uses its own bundled copy of the CLI library.
Each `patrol_mcp` release declares the `patrol_cli` version range it was
built and tested against.
| patrol\_mcp | patrol\_cli | patrol\_log |
| ----------- | ----------- | ----------- |
| 0.1.4 | 4.3.0+ | >=0.8.0 |
| 0.1.3 | 4.3.1 | ^0.8.0 |
| 0.1.2 | 4.3.0 | ^0.8.0 |
| 0.1.1 | 4.3.0 | ^0.8.0 |
| 0.1.0 | 4.3.0 | ^0.8.0 |
This table only matters if you have **both** `patrol_cli` and `patrol_mcp` in
the same `pubspec.yaml`. In that case, pub must resolve a single `patrol_cli`
version that satisfies both packages. If you hit a version conflict, use the
table above to find compatible versions.
## Getting Started [#getting-started]
See the [full setup guide on GitHub](https://github.com/leancodepl/patrol/blob/master/packages/patrol_mcp/README.md)
for installation steps, IDE-specific MCP configuration (Claude Code, Cursor,
Copilot, Gemini CLI), environment variables, and troubleshooting.
# Patrol tags
Patrol tags allow you to organize and selectively run your patrol tests. You can assign tags to individual tests and then use those tags to filter which tests to run or exclude.
By design, patrol tags should work the same as flutter test tags.
## Defining tags in tests [#defining-tags-in-tests]
You can assign tags to your Patrol tests using the `tags` parameter. Tags are defined as a list of strings:
```dart title="patrol_test/example_test.dart"
import 'package:flutter/material.dart';
import 'package:patrol/patrol.dart';
void main() {
patrolTest(
'short test with two tags',
tags: ['smoke', 'regression'],
($) async {
await createApp($);
await $(FloatingActionButton).tap();
expect($(#counterText).text, '1');
await $(FloatingActionButton).tap();
expect($(#counterText).text, '2');
},
);
patrolTest(
'short test with tag',
tags: ['smoke'],
($) async {
await createApp($);
await $(FloatingActionButton).tap();
expect($(#counterText).text, '1');
await $(#textField).enterText('Hello, Flutter!');
expect($('Hello, Flutter!'), findsOneWidget);
},
);
}
```
## Running tests with tags [#running-tests-with-tags]
Use the `--tags` option to run only tests that have specific tags:
```bash
# Run tests with the 'smoke' tag
patrol test --tags smoke
# Run tests with either 'smoke' or 'regression' tag
patrol test --tags='smoke||regression'
# Run tests with both 'login' and 'smoke' tags
patrol test --tags='(login && smoke)'
```
## Excluding tests with tags [#excluding-tests-with-tags]
Use the `--exclude-tags` option to exclude tests that have specific tags:
```bash
# Exclude tests with the 'regression' tag
patrol test --exclude-tags regression
# Exclude tests with either 'smoke' or 'regression' tag
patrol test --exclude-tags='(smoke||regression)'
```
## Tag expression syntax [#tag-expression-syntax]
Patrol supports complex tag expressions using logical operators:
### Basic operators [#basic-operators]
* `||` - OR operator (run tests with either tag)
* `&&` - AND operator (run tests with both tags)
* `!` - NOT operator (exclude tests with this tag)
Note that tags must be valid Dart identifiers, although they may also contain hyphens.
For more information about tag rules, see: [https://pub.dev/packages/test#tagging-tests](https://pub.dev/packages/test#tagging-tests)
### Examples [#examples]
```bash
# Run tests that have either 'smoke' OR 'regression' tag
patrol test --tags='smoke||regression'
# Run tests that have BOTH 'login' AND 'smoke' tags
patrol test --tags='(login && smoke)'
# Run tests with 'payment' OR 'navigation' tag, but NOT 'regression'
patrol test --tags='(payment || navigation) && !regression'
# Combine --tags with --exclude-tags
patrol test --tags='smoke||regression' --exclude-tags='slow'
# Complex expression: (login OR payment) AND (smoke OR regression)
patrol test --tags='((login || payment) && (smoke || regression))'
```
## Combining with other options [#combining-with-other-options]
You can combine tag filtering with other Patrol CLI options:
```
patrol test --target patrol_test/login_test.dart --tags smoke
```
# Guide for Patrol VS Code extension
## How to setup [#how-to-setup]
Install the extension from [VS Code Marketplace](https://marketplace.visualstudio.com/items?itemName=LeanCode.patrol-vscode)
or [Open VSX Registry for other VSCode-like editors, eg. Cursor](https://open-vsx.org/extension/LeanCode/patrol-vscode).
It requires Dart extension - if you don't have it, you'll be prompted to install it.
This step is only required if your tests are not located in the `patrol_test` directory.
In your project's pubspec.yaml, add a new line to patrol section:
```yaml title=pubspec.yaml
patrol:
test_directory: # default: patrol_test
```
Set this value to the directory where your patrol tests are located. Default value for `test_directory` is `patrol_test`.
If you have a method wrapping `patrolTest()`, you need to add an annotation to it:
```dart title=patrol_wrapper.dart
import 'package:meta/meta.dart';
@isTest // add this annotation here
void patrolWrapper(
String testName, Future Function(PatrolIntegrationTester) test) {
patrolTest(testName, test);
}
```
The `@isTest` annotation comes from the `meta` package. If `meta` is not in your `pubspec.yaml`, add it as a dev dependency:
```
flutter pub add meta --dev
```
You should now see patrol tests in the Test Explorer tab in VS Code. They should
be in Patrol section. Also you should see a green play button next to
`patrolTest` method invocation (or your wrapper invocation). See a screenshot below for a reference.
## Features [#features]
Let's go through features of our extension, that can help you developing and running Patrol tests.
### Test Explorer [#test-explorer]
Test Explorer is a tab on the left sidebar of VS Code. You can find there a list
of tests discovered in the opened project. You should see there all kinds of
tests in the project - unit, widget and integration - which comes from the Dart
extension. Our extension adds a section with Patrol tests.
You can use it to:
* run a single test file with the play button (1) next to test file name
* run all tests with play button at the top (2)
* debug a single test file (3)
You can see logs and result of the latest test run started through VS Code in
Test results tab (4). You'll also see there live logs from the current tests execution.
Running and debugging use the device chosen in VS Code.
The test execution can be stopped using stop button at the top. (5)
Keep in mind that running and debugging tests use commands from `patrol_cli` under
the hood, and you still have to have `patrol_cli` installed on your machine - activated
globally through pub or added to the project as a dependency in `pubspec.yaml`.
### Running tests from file [#running-tests-from-file]
Now you can run the tests by clicking the green play button beside first line of the test, as shown on the screenshot below.
Be aware that if you have more than one test in the file, this button will run the whole test file.
### Debugging tests [#debugging-tests]
We combined Patrol's `develop` command with debugging feature of VS Code - now
you can debug your tests easier in our extension!
To start debugging, click on debug icon in the Test explorer tab (as shown on
screenshot above, by no. 3). It will start the test in `develop` mode. This
means that:
* only one test file can be debugged at once - do not use the button
to debug all tests (available at the very top, works only for widget and unit
tests),
* debugging will continue after the test is finished. This allows you to
hot restart the test.
After the test is built, it will start executing and debugger inside VS Code
will attach to it. It takes a while, you'll see a debugging toolbar when it's ready.
The buttons on the toolbar are from the left:
* 4 buttons to navigate debugging (pause/continue, step over, step into, step out)
* 2 buttons that don't work - they do nothing in this mode and are added by Flutter extension
* disconnect button, which disconnects the debugger but leave the test running
* hot restart button - use it to hot restart the test
* stop button - stops the test and closes the app on the device
* Widget inspector button - opens Widget inspector inside VS Code. More about
devtools in general in the next section.
### Devtools [#devtools]
You can open devtools in many ways:
* from a popup that appears at the start of debugging session (1)
* from command palette (2)
Both those ways lead to this dropdown, where you can choose the tab of devtools
that you want to open or choose to open them in web browser.
Patrol's devtools extension which allows you to inspect native elements
tree is available only in web version of devtools.
### Additional arguments to patrol commands [#additional-arguments-to-patrol-commands]
Since test execution through the extension is using patrol\_cli commands under the hood,
you may want to pass more arguments to those commands beside the target file and the device.
You can modify those in VS Code settings.
## Troubleshooting [#troubleshooting]
Reload the window: open Command palette > Show and Run Commands > Developer: Reload Window.
Enable "Show implementation widgets" switch placed on the top bar of Widget inspector.
Make sure you completed the setup from the first section of this page.
It's a bug in some versions of VS Code resulting in PATH env var not being imported. Update your VS Code to the newest version.
# Tips and tricks
### Inspecting native view hierarchy [#inspecting-native-view-hierarchy]
It's hard to tap on or enter text into a view you don't know how to refer to. In
such situation we recommend doing a native view hierarchy dump and finding the
properties of the view you want to act on.
**Android**
First, perform a native view hierarchy dump using `adb`:
```
adb shell uiautomator dump
```
Then, copy the dump file from the device to your machine:
```
adb pull /sdcard/window_dump.xml .
```
**iOS**
The easiest way to perform the native view hierarchy dumb on iOS is to use the
[idb] tool.
Once you have [idb] installed, perform a dump:
```
idb ui describe-all
```
### Configuring test directory [#configuring-test-directory]
By default, Patrol looks for tests in the `patrol_test/` directory.
This default was changed from `integration_test/` to avoid conflicts with Flutter's official integration testing plugin
and to give Patrol tests their own dedicated space. This change was introduced in Patrol 4.0.0.
#### Using custom test directory [#using-custom-test-directory]
You can configure Patrol to use a different directory by adding `test_directory` to your `pubspec.yaml`:
```yaml title="pubspec.yaml"
patrol:
app_name: My App
test_directory: my_custom_tests # Custom directory
android:
package_name: com.example.myapp
ios:
bundle_id: com.example.MyApp
```
#### Migrating from integration\_test directory [#migrating-from-integration_test-directory]
If you have existing Patrol tests in the `integration_test/` directory, you have two options:
**Option 1: Rename integration\_test directory to patrol\_test**
**Option 2: Configure Patrol to use integration\_test**
```yaml title="pubspec.yaml"
patrol:
app_name: My App
test_directory: integration_test # Keep using old directory
android:
package_name: com.example.myapp
ios:
bundle_id: com.example.MyApp
```
Non-patrol integration tests should remain in the integration\_test directory.
### Avoiding hardcoding credentials in tests [#avoiding-hardcoding-credentials-in-tests]
It's a bad practice to hardcode data such as emails, usernames, and passwords in
test code.
```dart
await $(#nameTextField).enterText('Bartek'); // bad!
await $(#passwordTextField).enterText('ny4ncat'); // bad as well!
```
To fix this, we recommend removing the hardcoded credentials from test code and
providing them through the environment:
```dart
await $(#nameTextField).enterText(const String.fromEnvironment('USERNAME'));
await $(#passwordTextField).enterText(const String.fromEnvironment('PASSWORD'));
```
> Make sure that you're using `const` here because of [issue #55870][55870].
To set `USERNAME` and `PASSWORD`, use `--dart-define`:
```
patrol test --dart-define 'USERNAME=Bartek' --dart-define 'PASSWORD=ny4ncat'
```
Alternatively you can create a `.patrol.env` file in your project's root. Comments
are supported using the `#` symbol and can be inline or on their own line. Here's
an example:
```
$ cat .patrol.env
# Add your username here
EMAIL=user@example.com
PASSWORD=ny4ncat # The password for the API
```
### Granting sensitive permission through the Settings app [#granting-sensitive-permission-through-the-settings-app]
Some particularly sensitive permissions (such as access to background location
or controlling the Do Not Disturb feature) cannot be requested in the permission
dialog like most of the common permissions. Instead, you have to ask the user to
go to the Settings app and grant your app the permission you need.
Testing such flows is not as simple as simply granting normal permission, but
it's totally possible with Patrol.
Below we present you with a snippet that will make the built-in Camera app have
access to the Do Not Disturb feature on Android. Let's assume that the Settings
app on the device we want to run the tests on looks like this:
And here's the code:
```dart
await $.platform.mobile.tap(Selector(text: 'Camera')); // tap on the list tile
await $.platform.mobile.tap(Selector(text: 'ALLOW')); // handle the confirmation dialog
await $.platform.mobile.pressBack(); // go back to the app under test
```
Please note that the UI of the Settings app differs across operating systems,
their versions, and OEM flavors (in case of Android). You'll have to handle all
edge cases yourself.
### Ignoring exceptions [#ignoring-exceptions]
If an exception is thrown during a test, it is marked as failed. This is
Flutter's default behavior and it's usually good – after all, it's better to fix
the cause of a problem instead of ignoring it.
That said, sometimes you do have a legitimate reason to ignore an exception.
This can be accomplished with the help of the
[WidgetTester.takeException()][take_exception] method, which returns the last
exception that occurred and removes it from the internal list of uncaught
exceptions, so that it won't mark the test as failed. To use it, just call it
once:
```dart
final widgetTester = $.tester;
widgetTester.takeException();
```
If more than a single exception is thrown during the test and you want to ignore
all of them, the below snippet should come in handy:
```dart
var exceptionCount = 0;
dynamic exception = $.tester.takeException();
while (exception != null) {
exceptionCount++;
exception = $.tester.takeException();
}
if (exceptionCount != 0) {
$.log('Warning: $exceptionCount exceptions were ignored');
}
```
### Handling permission dialogs before the main app widget is pumped [#handling-permission-dialogs-before-the-main-app-widget-is-pumped]
Sometimes you might want to manually request permissions in the test before the
main app widget is pumped. Let's say that you're using the [geolocator] package:
```dart
final permission = await Geolocator.requestPermission();
final position = await Geolocator.getCurrentPosition();
await $.pumpWidgetAndSettle(MyApp(position: position));
```
In such case, first call the `requestPermission()` method, but instead of
awaiting it, assign the `Future` it returns to some `final`. Then, use Patrol to
grant the permissions, and finally, await the `Future` from the first step:
```dart
// 1. request the permission
final permissionRequestFuture = Geolocator.requestPermission();
// 2. grant the permission using Patrol
await $.platform.mobile.grantPermissionWhenInUse();
// 3. wait for permission being granted
final permissionRequestResult = await permissionRequestFuture;
expect(permissionRequestResult, equals(LocationPermission.whileInUse));
final position = await Geolocator.getCurrentPosition();
await $.pumpWidgetAndSettle(MyApp(position: position));
```
See also:
* [Patrol issue #628]
[patrol issue #628]: https://github.com/leancodepl/patrol/issues/628
[geolocator]: https://pub.dev/packages/geolocator
[idb]: https://github.com/facebook/idb
[take_exception]: https://api.flutter.dev/flutter/flutter_test/WidgetTester/takeException.html
[55870]: https://github.com/flutter/flutter/issues/55870
# Advanced
### Patrol section in `pubspec.yaml` [#patrol-section-in-pubspecyaml]
If your app has different name on iOS and Android, you can specify `app_name`
twice – one in `android` block, and one in `ios` block.
Though the whole Patrol section in `pubspec.yaml` is optional, we highly recommend
adding this section, because it enables the following features:
* Patrol will automatically uninstall your app after every test (using
`package_name` and `bundle_id`). This will make the environment which your
tests run in more stable and predictable.
* Patrol will be able to tap on your app's notifications (using `app_name`)
### Specific version of `patrol_cli` [#specific-version-of-patrol_cli]
You can install a specific version of Patrol CLI. For example:
```
dart pub global activate patrol_cli ^1.0.0
```
will install the latest `v1` version. We recommend to install a specific
version on CI systems to avoid unexpected breakages.
### Isolation of test runs [#isolation-of-test-runs]
To achieve full isolation between test runs:
* On Android: set `clearPackageData` to `true` in your `build.gradle` file,
* On iOS Simulator: use the `--full-isolation` flag
This functionality is experimental on iOS and might be removed in the future releases.
```bash
patrol test --full-isolation
```
#### Android gradle configuration [#android-gradle-configuration]
For Android, you can also set a default value by configuring `clearPackageData` in your app's `build.gradle` file:
```groovy title="android/app/build.gradle"
defaultConfig {
//...
testInstrumentationRunner "pl.leancode.patrol.PatrolJUnitRunner"
testInstrumentationRunnerArguments["clearPackageData"] = "true"
}
```
### Embrace the native tests [#embrace-the-native-tests]
If you've diligently followed the steps in [native automation setup] and `patrol test` prints a **TEST
PASSED** message, you might be now thinking: what did I just do?
The answer is: You've just integrated Flutter testing with native Android/iOS
testing frameworks. This means that your Flutter integration tests can now be
run as *native tests*.
#### What are native tests good for, anyway? [#what-are-native-tests-good-for-anyway]
iOS and Android have existed for more than 15 years, and during that time many
of awesome testing-related things were built around them – open-source test
runners, device farms, HTML report generators. Developers who create native
mobile apps can easily reap benefits from this huge, mature ecosystem.
Meanwhile we, Flutter developers, don't have as much at our disposal. Our
framework is much younger and less mature.
What if we could masquerade our Flutter tests so that from the outside they
would be truly native? This way we leverage many existing tools while
maintaining the convenience of writing the tests in pure Dart.
> For example, you can run your Patrol tests directly from Xcode. Xcode knows
> nothing about Flutter, Dart and Patrol – it only launches your test app.
> Flutter tests are then run inside the test app and the results are reported
> back to Xcode. This way you get the best of both worlds – the maturity of
> native iOS development and the productivity of Flutter and Dart.
That's exactly what Patrol does (and what the default [integration\_test][integration_test] package
does at well, but at a bit smaller scale).
Take a look at this simple Flutter integration tests using Patrol:
```dart title="patrol_test/example_test.dart"
void main() {
patrolTest(
'counter state is the same after going to Home and switching apps',
nativeAutomatorConfig: NativeAutomatorConfig(
packageName: 'pl.leancode.patrol.example',
bundleId: 'pl.leancode.patrol.Example',
),
($) async {
await $.pumpWidget(ExampleApp());
await $(FloatingActionButton).tap();
expect($(#counterText).text, '1');
await $.platform.mobile.pressHome();
await $.platform.mobile.openApp();
expect($(#counterText).text, '1');
await $(FloatingActionButton).tap();
expect($(#counterText).text, '2');
},
);
}
```
You can run this test and view its results in many ways, using all sorts of
different tools, platforms, and IDEs:
When Android test finishes, its test results are automatically generated in
`build/app/outputs/androidTest-results/connected/test-result.pb`. To view
them in Android Studio, use the `Run > Import tests from file` option.
It just works ✨
You don't have to use the bulky Android Studio to view your test results,
because Gradle 🐘 automatically generates a nice HTML summary!
You can find it in `build/app/reports/androidTests/connected/index.html`.
With the help of awesome [fastlane scan] you can prettify the output of
`xcodebuild` to make it easier to understand and generate HTML summary of
your tests.
This is so awesome!
[native automation setup]: /documentation
[fastlane scan]: https://docs.fastlane.tools/actions/scan
[integration_test]: https://github.com/flutter/flutter/tree/master/packages/integration_test
# Feature parity
Here you can see what you can already do with Patrol's `PlatformAutomator`, and what
is yet to be implemented. We hope that it will help you evaluate Patrol.
We strive for high feature parity across platforms, but in some cases it's
impossible to reach 100%. Web support is available for browser-specific automation.
macOS support is still in alpha and does not have platform automation implemented yet.
## Mobile features [#mobile-features]
These features are available via `$.platform.mobile` and work on both Android and iOS:
| **Feature** | **Android** | **iOS** |
| ------------------------------ | ------------ | --------------- |
| [Press home] | ✅ | ✅ |
| [Open app] | ✅ | ✅ |
| [Open notifications] | ✅ | ✅ |
| [Close notifications] | ✅ | ✅ |
| [Open quick settings] | ✅ | ✅ |
| [Open URL] | ✅ | ✅ |
| [Enable/disable dark mode] | ✅ | ✅ |
| [Enable/disable airplane mode] | ✅ | ✅ |
| [Enable/disable cellular] | ✅ | ✅ |
| [Enable/disable Wi-Fi] | ✅ | ✅ |
| [Enable/disable Bluetooth] | ✅ | ✅ |
| [Press volume up] | ✅ | ✅ (simulator ❌) |
| [Press volume down] | ✅ | ✅ (simulator ❌) |
| [Handle permission dialogs] | ✅ | ✅ |
| [Set mock location] | ✅ (device ❌) | ✅ |
| [Get OS version] | ✅ | ✅ |
| [Check virtual device] | ✅ | ✅ |
## Android-specific features [#android-specific-features]
These features are available via `$.platform.android`:
| **Feature** | **Android** |
| -------------------------- | ------------- |
| [Press back] | ✅ |
| [Double press recent apps] | ✅ |
| [Tap] | ✅ |
| [Double tap] | ✅ |
| [Tap at coordinate] | ✅ |
| [Enter text] | ✅ |
| [Enter text by index] | ✅ |
| [Swipe] | ✅ |
| [Swipe back] | ✅ |
| [Pull to refresh] | ✅ |
| [Tap on notification] | ✅ |
| [Enable/disable location] | ✅ |
| [Take camera photo] | ✅ |
| [Pick image from gallery] | ✅ |
| [Pick multiple images] | ✅ |
| Interact with WebView | ⚠️ see [#244] |
## iOS-specific features [#ios-specific-features]
These features are available via `$.platform.ios`:
| **Feature** | **iOS** |
| ----------------------------- | ------- |
| [iOS Tap] | ✅ |
| [iOS Double tap] | ✅ |
| [iOS Tap at coordinate] | ✅ |
| [iOS Enter text] | ✅ |
| [iOS Enter text by index] | ✅ |
| [iOS Swipe] | ✅ |
| [iOS Swipe back] | ✅ |
| [iOS Pull to refresh] | ✅ |
| [iOS Tap on notification] | ✅ |
| [Close heads-up notification] | ✅ |
| [iOS Take camera photo] | ✅ |
| [iOS Pick image from gallery] | ✅ |
| [iOS Pick multiple images] | ✅ |
| Interact with WebView | ✅ |
## Web-specific features [#web-specific-features]
These features are available via `$.platform.web` for Flutter Web apps:
| **Feature** | **Web** |
| -------------------------------- | ------- |
| [Web Tap] | ✅ |
| [Web Enter text] | ✅ |
| [Scroll to] | ✅ |
| [Enable/disable dark mode (web)] | ✅ |
| [Grant/clear permissions] | ✅ |
| [Manage cookies] | ✅ |
| [Upload files] | ✅ |
| [Handle dialogs] | ✅ |
| [Press key/key combo] | ✅ |
| [Browser navigation] | ✅ |
| [Clipboard operations] | ✅ |
| [Resize window] | ✅ |
| [Verify file downloads] | ✅ |
{/* Issue links */}
[#244]: https://github.com/leancodepl/patrol/issues/244
{/* MobileAutomator links */}
[Press home]: https://pub.dev/documentation/patrol/latest/patrol/MobileAutomator/pressHome.html
[Open app]: https://pub.dev/documentation/patrol/latest/patrol/MobileAutomator/openApp.html
[Open notifications]: https://pub.dev/documentation/patrol/latest/patrol/MobileAutomator/openNotifications.html
[Close notifications]: https://pub.dev/documentation/patrol/latest/patrol/MobileAutomator/closeNotifications.html
[Open quick settings]: https://pub.dev/documentation/patrol/latest/patrol/MobileAutomator/openQuickSettings.html
[Open URL]: https://pub.dev/documentation/patrol/latest/patrol/MobileAutomator/openUrl.html
[Enable/disable dark mode]: https://pub.dev/documentation/patrol/latest/patrol/MobileAutomator/enableDarkMode.html
[Enable/disable airplane mode]: https://pub.dev/documentation/patrol/latest/patrol/MobileAutomator/enableAirplaneMode.html
[Enable/disable cellular]: https://pub.dev/documentation/patrol/latest/patrol/MobileAutomator/enableCellular.html
[Enable/disable Wi-Fi]: https://pub.dev/documentation/patrol/latest/patrol/MobileAutomator/enableWifi.html
[Enable/disable Bluetooth]: https://pub.dev/documentation/patrol/latest/patrol/MobileAutomator/enableBluetooth.html
[Press volume up]: https://pub.dev/documentation/patrol/latest/patrol/MobileAutomator/pressVolumeUp.html
[Press volume down]: https://pub.dev/documentation/patrol/latest/patrol/MobileAutomator/pressVolumeDown.html
[Handle permission dialogs]: https://pub.dev/documentation/patrol/latest/patrol/MobileAutomator/grantPermissionWhenInUse.html
[Set mock location]: https://pub.dev/documentation/patrol/latest/patrol/MobileAutomator/setMockLocation.html
[Get OS version]: https://pub.dev/documentation/patrol/latest/patrol/MobileAutomator/getOsVersion.html
[Check virtual device]: https://pub.dev/documentation/patrol/latest/patrol/MobileAutomator/isVirtualDevice.html
{/* AndroidAutomator links */}
[Press back]: https://pub.dev/documentation/patrol/latest/patrol/AndroidAutomator/pressBack.html
[Double press recent apps]: https://pub.dev/documentation/patrol/latest/patrol/AndroidAutomator/pressDoubleRecentApps.html
[Tap]: https://pub.dev/documentation/patrol/latest/patrol/AndroidAutomator/tap.html
[Double tap]: https://pub.dev/documentation/patrol/latest/patrol/AndroidAutomator/doubleTap.html
[Tap at coordinate]: https://pub.dev/documentation/patrol/latest/patrol/AndroidAutomator/tapAt.html
[Enter text]: https://pub.dev/documentation/patrol/latest/patrol/AndroidAutomator/enterText.html
[Enter text by index]: https://pub.dev/documentation/patrol/latest/patrol/AndroidAutomator/enterTextByIndex.html
[Swipe]: https://pub.dev/documentation/patrol/latest/patrol/AndroidAutomator/swipe.html
[Swipe back]: https://pub.dev/documentation/patrol/latest/patrol/AndroidAutomator/swipeBack.html
[Pull to refresh]: https://pub.dev/documentation/patrol/latest/patrol/AndroidAutomator/pullToRefresh.html
[Tap on notification]: https://pub.dev/documentation/patrol/latest/patrol/AndroidAutomator/tapOnNotificationBySelector.html
[Enable/disable location]: https://pub.dev/documentation/patrol/latest/patrol/AndroidAutomator/enableLocation.html
[Take camera photo]: https://pub.dev/documentation/patrol/latest/patrol/AndroidAutomator/takeCameraPhoto.html
[Pick image from gallery]: https://pub.dev/documentation/patrol/latest/patrol/AndroidAutomator/pickImageFromGallery.html
[Pick multiple images]: https://pub.dev/documentation/patrol/latest/patrol/AndroidAutomator/pickMultipleImagesFromGallery.html
{/* IOSAutomator links */}
[iOS Tap]: https://pub.dev/documentation/patrol/latest/patrol/IOSAutomator/tap.html
[iOS Double tap]: https://pub.dev/documentation/patrol/latest/patrol/IOSAutomator/doubleTap.html
[iOS Tap at coordinate]: https://pub.dev/documentation/patrol/latest/patrol/IOSAutomator/tapAt.html
[iOS Enter text]: https://pub.dev/documentation/patrol/latest/patrol/IOSAutomator/enterText.html
[iOS Enter text by index]: https://pub.dev/documentation/patrol/latest/patrol/IOSAutomator/enterTextByIndex.html
[iOS Swipe]: https://pub.dev/documentation/patrol/latest/patrol/IOSAutomator/swipe.html
[iOS Swipe back]: https://pub.dev/documentation/patrol/latest/patrol/IOSAutomator/swipeBack.html
[iOS Pull to refresh]: https://pub.dev/documentation/patrol/latest/patrol/IOSAutomator/pullToRefresh.html
[iOS Tap on notification]: https://pub.dev/documentation/patrol/latest/patrol/IOSAutomator/tapOnNotificationBySelector.html
[Close heads-up notification]: https://pub.dev/documentation/patrol/latest/patrol/IOSAutomator/closeHeadsUpNotification.html
[iOS Take camera photo]: https://pub.dev/documentation/patrol/latest/patrol/IOSAutomator/takeCameraPhoto.html
[iOS Pick image from gallery]: https://pub.dev/documentation/patrol/latest/patrol/IOSAutomator/pickImageFromGallery.html
[iOS Pick multiple images]: https://pub.dev/documentation/patrol/latest/patrol/IOSAutomator/pickMultipleImagesFromGallery.html
{/* WebAutomator links */}
[Web Tap]: https://pub.dev/documentation/patrol/latest/patrol/WebAutomator/tap.html
[Web Enter text]: https://pub.dev/documentation/patrol/latest/patrol/WebAutomator/enterText.html
[Scroll to]: https://pub.dev/documentation/patrol/latest/patrol/WebAutomator/scrollTo.html
[Enable/disable dark mode (web)]: https://pub.dev/documentation/patrol/latest/patrol/WebAutomator/enableDarkMode.html
[Grant/clear permissions]: https://pub.dev/documentation/patrol/latest/patrol/WebAutomator/grantPermissions.html
[Manage cookies]: https://pub.dev/documentation/patrol/latest/patrol/WebAutomator/addCookie.html
[Upload files]: https://pub.dev/documentation/patrol/latest/patrol/WebAutomator/uploadFile.html
[Handle dialogs]: https://pub.dev/documentation/patrol/latest/patrol/WebAutomator/acceptNextDialog.html
[Press key/key combo]: https://pub.dev/documentation/patrol/latest/patrol/WebAutomator/pressKey.html
[Browser navigation]: https://pub.dev/documentation/patrol/latest/patrol/WebAutomator/goBack.html
[Clipboard operations]: https://pub.dev/documentation/patrol/latest/patrol/WebAutomator/getClipboard.html
[Resize window]: https://pub.dev/documentation/patrol/latest/patrol/WebAutomator/resizeWindow.html
[Verify file downloads]: https://pub.dev/documentation/patrol/latest/patrol/WebAutomator/verifyFileDownloads.html
# Native Automation 2.0 (native2)
`native2` is deprecated starting from Patrol version `4.0.0` and will be removed in a future release.
Please migrate to the new [Platform Automation API](/documentation/native/overview) using `$.platform.mobile` instead.
`native2` is available starting from Patrol version `3.6.0`.
## What is `native2`? [#what-is-native2]
`native2` was created to address fundamental limitations in the original native automation approach. The original native API was primarily designed for Android, and attempts to make a single `Selector` work across both
platforms proved problematic because **iOS and Android use different selector arguments** (eg. Android's `resourceName` vs iOS's `identifier`) and a single selector approach couldn't effectively handle the fundamental differences
between iOS and Android element identification. **`native2` provides platform-specific selectors that work with both Android and iOS, giving you more accurate selectors instead of one shared selector.**
### Before `native2` [#before-native2]
```dart
// You were forced to use flaky text-based selectors that work on both platforms
await $.native.tap(Selector(textContains: 'Login'));
```
```dart
// Before: Sometimes you needed to use platform-specific if statements in your test code.
if (Platform.isAndroid) {
await $.native.tap(Selector(resourceId: 'com.android.camera2:id/shutter_button'));
} else {
await $.native.tap(Selector(text: 'Take Picture'));
}
```
### With `native2` [#with-native2]
`native2` provides a single method call that works across both platforms:
```dart
// After: Single method call with platform-specific selectors
await $.native2.tap(
NativeSelector(
android: AndroidSelector(
resourceName: 'com.android.camera2:id/shutter_button',
),
ios: IOSSelector(label: 'Take Picture'),
),
);
```
### Text Input Operations [#text-input-operations]
```dart
// Enter password in secure field
await $.native2.enterText(
NativeSelector(
android: AndroidSelector(
contentDescription: 'Password',
),
ios: IOSSelector(
elementType: IOSElementType.secureTextField,
),
),
text: 'secretpassword',
);
```
### More platform-specific attributes like elementType for iOS [#more-platform-specific-attributes-like-elementtype-for-ios]
```dart
// Find elements by instance (when multiple elements match)
await $.native2.tap(
NativeSelector(
android: AndroidSelector(
className: 'android.widget.Button',
instance: 2, // Third button (0-indexed)
),
ios: IOSSelector(
elementType: IOSElementType.button,
instance: 2, // Third button (0-indexed)
),
),
);
```
### Specifying App ID (iOS) [#specifying-app-id-ios]
When working with iOS, you may need to specify the `appId` parameter to interact with elements in specific applications. This is particularly useful when your test needs to interact with system apps like Safari, Settings, or other third-party applications.
The `appId` parameter is used for iOS. On Android, it will be ignored.
```dart
await $.native2.tap(
appId: 'com.apple.mobilesafari',
NativeSelector(
ios: IOSSelector(elementType: IOSElementType.button, label: 'Open'),
),
);
```
Remember that if you don't provide a platform-specific selector (iOS or Android) and run the command on that platform, the command will fail.
# Overview
Flutter's [integration\_test][integration_test] does a good job at providing
basic support for integration testing Flutter apps. What it can't do is
interaction with the OS your Flutter app is running on. This makes it impossible
to test many critical business features:
* granting runtime permissions
* signing into the app which uses WebView or OAuth (like Google) as the login
page
* listing and tapping on notifications
* exiting the app, coming back, and verifying that state is preserved
* enabling and disabling features such as Wi-Fi, mobile data, location, or dark
mode
Patrol's *platform automation* feature finally solves these problems. Here's a
tiny snippet to spice things up:
```dart title="patrol_test/demo_test.dart"
void main() {
patrolTest('demo', (PatrolIntegrationTester $) async {
await $.pumpWidgetAndSettle(AwesomeApp());
// prepare network conditions
await $.platform.mobile.enableCellular();
await $.platform.mobile.disableWifi();
// toggle system theme
await $.platform.mobile.enableDarkMode();
// handle native location permission request dialog
await $.platform.mobile.selectFineLocation();
await $.platform.mobile.grantPermissionWhenInUse();
// tap on the first notification
await $.platform.mobile.openNotifications();
await $.platform.mobile.tapOnNotificationByIndex(0);
});
}
```
For web applications, Patrol provides browser-specific automation capabilities. You can
interact with browser dialogs, manage cookies, handle file uploads, and more:
```dart title="patrol_test/web_demo_test.dart"
void main() {
patrolTest('web demo', (PatrolIntegrationTester $) async {
await $.pumpWidgetAndSettle(AwesomeWebApp());
// grant browser permissions
await $.platform.web.grantPermissions(permissions: ['clipboard-read']);
// manage cookies
await $.platform.web.addCookie(name: 'session', value: 'abc123');
// handle browser dialogs
await $.platform.web.acceptNextDialog();
});
}
```
Platform automation is currently available on Android, iOS, and Web.
[integration_test]: https://github.com/flutter/flutter/tree/master/packages/integration_test
# Usage
Once set up, interacting with the native UI using Patrol is very easy!
### Basics [#basics]
After you've got your `PlatformAutomator` object via `$.platform`, you simply call methods on it
and it does the magic. For cross-platform mobile actions, use `$.platform.mobile`.
To tap on a native view (for example, a button in a WebView):
```dart
await $.platform.mobile.tap(Selector(text: 'Sign up for newsletter'));
```
To enter text into a native view (for example, a form in a WebView):
```dart
await $.platform.mobile.enterText(
Selector(text: 'Enter your email'),
text: 'charlie@root.me',
);
```
You can also enter text into the n-th currently visible text field (counting from 0):
```dart
await $.platform.mobile.enterTextByIndex('charlie_root', index: 0); // enter username
await $.platform.mobile.enterTextByIndex('ny4ncat', index: 1); // enter password
```
The above are the simplest, most common actions, but they already make it
possible to test scenarios that were impossible to test before, such as
WebViews.
### Platform-specific selectors [#platform-specific-selectors]
When you need different selectors for Android and iOS, use `MobileSelector`:
```dart
await $.platform.mobile.tap(
MobileSelector(
android: AndroidSelector(resourceName: 'com.example:id/button'),
ios: IOSSelector(identifier: 'myButton'),
),
);
```
### Android-specific actions [#android-specific-actions]
For Android-only actions, use `$.platform.android`:
```dart
// Press the hardware back button (Android only)
await $.platform.android.pressBack();
// Double press the recent apps button to switch to the previous app
await $.platform.android.pressDoubleRecentApps();
// Tap on a native view using Android-specific selector
await $.platform.android.tap(
AndroidSelector(resourceName: 'com.example:id/submit_button'),
);
// Open a specific Android app
await $.platform.android.openPlatformApp(androidAppId: 'com.android.settings');
```
### iOS-specific actions [#ios-specific-actions]
For iOS-only actions, use `$.platform.ios`:
```dart
// Tap on a native view using iOS-specific selector
await $.platform.ios.tap(
IOSSelector(identifier: 'submitButton'),
);
// Close a heads-up notification
await $.platform.ios.closeHeadsUpNotification();
// Swipe back gesture (iOS edge swipe)
await $.platform.ios.swipeBack();
// Open a specific iOS app
await $.platform.ios.openPlatformApp(iosAppId: 'com.apple.Preferences');
```
To tap, enter text, or perform generally any UI interaction with an iOS app
that is not your Flutter app under test, you need to pass its bundle
identifier. For example, to tap on the "Add" button in the iPhone contacts app:
```dart
await $.platform.ios.tap(
IOSSelector(text: 'Add'),
appId: 'com.apple.MobileAddressBook',
);
```
### Web-specific actions [#web-specific-actions]
For Flutter Web apps, use `$.platform.web` to interact with browser elements and features:
```dart
// Tap on a web element by text
await $.platform.web.tap(WebSelector(text: 'Submit'));
// Tap on a web element using CSS selector
await $.platform.web.tap(WebSelector(cssOrXpath: 'css=#submit-button'));
// Tap on a web element by test ID
await $.platform.web.tap(WebSelector(testId: 'login-button'));
// Enter text into a form field
await $.platform.web.enterText(
WebSelector(placeholder: 'Email address'),
text: 'user@example.com',
);
// Scroll to an element
await $.platform.web.scrollTo(WebSelector(text: 'Load more'));
```
Web automation also supports advanced browser interactions:
```dart
// Handle browser dialogs
await $.platform.web.acceptNextDialog();
await $.platform.web.dismissNextDialog();
// Manage cookies
await $.platform.web.addCookie(name: 'session', value: 'abc123');
await $.platform.web.clearCookies();
// Control dark mode
await $.platform.web.enableDarkMode();
await $.platform.web.disableDarkMode();
// Browser navigation
await $.platform.web.goBack();
await $.platform.web.goForward();
// Keyboard interactions
await $.platform.web.pressKey(key: 'Enter');
await $.platform.web.pressKeyCombo(keys: ['Control', 'a']);
// Clipboard operations
await $.platform.web.setClipboard(text: 'Copied text');
final clipboardContent = await $.platform.web.getClipboard();
// Browser permissions
await $.platform.web.grantPermissions(permissions: ['geolocation', 'notifications']);
await $.platform.web.clearPermissions();
// File uploads
await $.platform.web.uploadFile(files: [UploadFileData(name: 'test.txt', content: 'Hello')]);
// Verify file downloads
final downloadedFiles = await $.platform.web.verifyFileDownloads();
// Resize browser window
await $.platform.web.resizeWindow(size: Size(1920, 1080));
```
Working with iframes:
```dart
// Tap on an element inside an iframe
await $.platform.web.tap(
WebSelector(text: 'Submit'),
iframeSelector: WebSelector(cssOrXpath: 'css=#payment-iframe'),
);
```
### Cross-platform mobile actions [#cross-platform-mobile-actions]
For actions that work on both Android and iOS, use `$.platform.mobile`. This is the recommended
approach when you don't need platform-specific behavior, as it keeps your tests clean and
maintainable across both platforms.
The `$.platform.mobile` automator automatically routes calls to the appropriate platform
implementation based on where your test is running. You can use the unified `Selector` class
for simple cases, or `MobileSelector` when you need different selectors per platform.
### Notifications [#notifications]
To open the notification shade:
```dart
await $.platform.mobile.openNotifications();
```
To tap on the second notification:
```dart
await $.platform.mobile.tapOnNotificationByIndex(1);
```
You can also tap on notification by its content:
```dart
await $.platform.mobile.tapOnNotificationBySelector(
Selector(textContains: 'Someone liked your recent post'),
);
```
### Permissions [#permissions]
To handle the native permission request dialog:
```dart
await $.platform.mobile.grantPermissionWhenInUse();
await $.platform.mobile.grantPermissionOnlyThisTime();
await $.platform.mobile.denyPermission();
```
If the permission request dialog visible is the location dialog, you can also
select the accuracy:
```dart
await $.platform.mobile.selectFineLocation();
await $.platform.mobile.selectCoarseLocation();
```
The test will fail if the permission request dialog is not visible. You can
check if it is with:
```dart
if (await $.platform.mobile.isPermissionDialogVisible()) {
await $.platform.mobile.grantPermissionWhenInUse();
}
```
By default, `isPermissionDialogVisible()` waits for a short amount of time and
then returns `false` if the dialog is not visible. To increase the timeout:
```dart
if (await $.platform.mobile.isPermissionDialogVisible(timeout: Duration(seconds: 5))) {
await $.platform.mobile.grantPermissionWhenInUse();
}
```
Patrol can handle permissions on iOS only if the device language is set to
English (preferably US). That's because there's no way to refer to a specific
view in a language-independent way (like resourceId on Android).
If you want to handle permissions on iOS device with non-English locale, do it
manually:
```dart
await $.platform.ios.tap(
IOSSelector(text: 'Allow'),
appId: 'com.apple.springboard',
);
```
### Device information [#device-information]
Get information about the device running the tests:
```dart
// Check if running on an emulator/simulator
final isVirtual = await $.platform.mobile.isVirtualDevice();
// Get OS version (e.g., 30 for Android 11)
final osVersion = await $.platform.mobile.getOsVersion();
if (osVersion >= 30) {
// Android 11+ specific behavior
}
```
### More resources [#more-resources]
To see more integration tests demonstrating Patrol's various features, check out
our [example app][example_app].
[example_app]: https://github.com/leancodepl/patrol/tree/master/packages/patrol/example
