BrowserStack

Setup

BrowserStack App Automate is a popular cloud device farm. You can use it to run your tests on real devices.

You can choose between running tests in a recommended way using scripts or manually:

After scheduling the test execution, you can check the status of the test execution in the App Automate dashboard.

If you need to change the test configuration, check out full list of available devices and OS versions in the BrowserStack Browsers & Devices.

Sharding

BrowserStack runs each shard on its own device, in parallel. The two platforms shard differently.

Android

Espresso shards split automatically: add shards to the build request and BrowserStack distributes the test classes across that many devices.

"shards": { "numberOfShards": 3 }

This works with runtime discovery (every test is a parameter of your host test class) and with build-time discovery (every Dart test file is its own class). BrowserStack balances by test count, not duration, so a shard that got the slow tests can still hit the session cap - add shards rather than expecting an even split.

iOS

XCUITest has no automatic split. BrowserStack needs an explicit list of test identifiers per shard, and there is nothing to list under runtime discovery: the tests are registered only after the app launches. Enable build-time test discovery first - each Dart test then compiles to a RunnerUITests/PatrolGeneratedTests_<file>/test_<name> method that you can hand to BrowserStack's only-testing strategy.

Generate the identifiers from the .inc the build wrote, so they stay byte-identical to what XCTest discovers, and distribute them round-robin:

# After `patrol build ios` with emit_test_manifest enabled.
ids=$(awk '/^@implementation /{cls=$2} /^- \(void\)test_/{m=$2; sub(/^\(void\)/,"",m); \
  print "RunnerUITests/" cls "/" m}' ios/RunnerUITests/PatrolGeneratedTests.inc)

shards=3
shards_json=$(printf '%s\n' "$ids" | jq -R . | jq -s --argjson n "$shards" '{
  numberOfShards: $n,
  mapping: [range(0; $n) as $s | {
    name: ($s | tostring),
    strategy: "only-testing",
    values: [to_entries[] | select(.key % $n == $s) | .value]
  }]
}')

Then pass it in the xctestrun-build request next to the app and test suite:

"shards": <shards_json>,
"singleRunnerInvocation": true