BrowserStack
Setup
BrowserStack App Automate is a popular cloud device farm. You can use it to run your tests on real devices.
You can choose between running tests in a recommended way using scripts or manually:
After scheduling the test execution, you can check the status of the test execution in the App Automate dashboard.
If you need to change the test configuration, check out full list of available devices and OS versions in the BrowserStack Browsers & Devices.
Sharding
BrowserStack runs each shard on its own device, in parallel. The two platforms shard differently.
Android
Espresso shards split automatically: add shards to the build request and
BrowserStack distributes the test classes across that many devices.
"shards": { "numberOfShards": 3 }This works with runtime discovery (every test is a parameter of your host test class) and with build-time discovery (every Dart test file is its own class). BrowserStack balances by test count, not duration, so a shard that got the slow tests can still hit the session cap - add shards rather than expecting an even split.
iOS
XCUITest has no automatic split. BrowserStack needs an explicit list of test
identifiers per shard, and there is nothing to list under runtime discovery: the
tests are registered only after the app launches. Enable
build-time test discovery first -
each Dart test then compiles to a RunnerUITests/PatrolGeneratedTests_<file>/test_<name>
method that you can hand to BrowserStack's only-testing strategy.
Generate the identifiers from the .inc the build wrote, so they stay
byte-identical to what XCTest discovers, and distribute them round-robin:
# After `patrol build ios` with emit_test_manifest enabled.
ids=$(awk '/^@implementation /{cls=$2} /^- \(void\)test_/{m=$2; sub(/^\(void\)/,"",m); \
print "RunnerUITests/" cls "/" m}' ios/RunnerUITests/PatrolGeneratedTests.inc)
shards=3
shards_json=$(printf '%s\n' "$ids" | jq -R . | jq -s --argjson n "$shards" '{
numberOfShards: $n,
mapping: [range(0; $n) as $s | {
name: ($s | tostring),
strategy: "only-testing",
values: [to_entries[] | select(.key % $n == $s) | .value]
}]
}')Then pass it in the xctestrun-build request next to the app and test suite:
"shards": <shards_json>,
"singleRunnerInvocation": trueThe identifier format is RunnerUITests/<class>/<method> with slashes on
BrowserStack real devices - unlike SauceLabs, which wants a dot between the
target and the class there. An identifier that matches nothing does not fail
the build: the shard installs the runner and sits on a blank screen until
BrowserStack's idle timeout kills it. Regenerate the list on every build,
because renaming a test or moving it to another file changes its identifier.
Don't request more shards than you have tests - BrowserStack still boots a device for an empty shard. A single test needs no sharding at all.
Dart code coverage (Android)
Patrol can collect Dart line coverage on BrowserStack and hand it back through BrowserStack's Espresso coverage API.
Nothing on the device side is BrowserStack-specific: the same setup works on other Android device farms, see emulator.wtf and Firebase Test Lab.
Instrument the app under test: in android/app/build.gradle.kts set enableAndroidTestCoverage = true (AGP 8+; testCoverageEnabled = true on AGP 7) on the debug build type.
Build with --dart-define=PATROL_BS_COVERAGE=true and --dart-define=PATROL_BS_COVERAGE_PACKAGES=^package:my_app/ (comma-separate several prefixes).
With PATROL_BS_COVERAGE_PACKAGES set to plain anchored prefixes, Patrol force-compiles your own libraries before collecting, so never-executed lines count towards the total, the same way flutter test --coverage counts them. The prefixes are handed to the VM as library filters, so only your code is compiled, and only once per app process.
Override the default with --dart-define=PATROL_BS_COVERAGE_FORCE_COMPILE=false to report only the code the suite executed. It is the cheapest mode, and the percentage is relative to loaded code, not the whole app.
Leave PATROL_BS_COVERAGE_PACKAGES unset, or use regexp syntax such as ^package:(a|b)/, and the filter cannot be passed to the VM. Patrol then falls back to executed-code-only coverage, because force-compiling without a filter walks the whole program, Flutter framework and SDK included, on every collection — minutes per test instead of seconds. Forcing it anyway with PATROL_BS_COVERAGE_FORCE_COMPILE=true logs a warning.
With useOrchestrator:true every test runs in a fresh process, so the compile is paid per test rather than once per shard. For large suites, prefer useOrchestrator:false or add shards.
Start the build with "coverage": "true". useOrchestrator can be either value and sharding works. clearPackageData:true works too, with useOrchestrator:true (see the note below).
Pull and merge the result once the build finishes:
patrol coverage pull --browserstack-build-id <build_id> --output coverageCredentials come from --browserstack-creds USERNAME:ACCESS_KEY or the BROWSERSTACK_CREDS env var. This writes coverage/jacoco.exec (Java) and coverage/patrol_lcov.info (Dart, merged across all shards).
With clearPackageData:true, pm clear between tests revokes the app's access to the shared coverage file. Patrol then appends each test's coverage as the shell user instead, and patrol coverage pull merges the per-test runs. This needs Android 12+ (API 31) devices; on older ones only the first test's coverage is kept and patrol coverage pull fails for that session, so use clearPackageData:false there.