LambdaTest (TestMu AI)

LambdaTest App Test Automation, now renamed to TestMu AI, is a popular cloud device farm. This page shows how to run Patrol tests on it:

Android: switch to LambdaTestPatrolJUnitRunner and upload two APKs.
iOS (real devices): enable build-time test discovery so each test runs in its own session.

Android

Change runner

Modify the app-level build.gradle:

android/app/build.gradle

android {
  // ...
  defaultConfig {
    //...
    testInstrumentationRunner "pl.leancode.patrol.LambdaTestPatrolJUnitRunner"
  }
  // ...
}

// ...

Upload to LambdaTest

To run Android UI tests on LambdaTest:

Upload the app under test APK to LambdaTest (see docs)
Upload the instrumentation app APK to LambdaTest (see docs)
Start test execution on LambdaTest (see docs)

$ export LAMBDATEST_PROJECT=AwesomeApp # optional
$ export LAMBDATEST_DEVICES="[\"Pixel 7 Pro-13\"]" # optional
• Building apk with entrypoint test_bundle.dart...
✓ Completed building apk with entrypoint test_bundle.dart (11.0s)
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 82.4M  100   255  100 82.4M      7  2897k  0:00:31  0:00:36  0:00:02 2051k
Uploaded app, "app_id": "lt://APP1016047291733313441063634",
Uploaded testsuite, "app_id": "lt://APP1016047291733312896265135",
{
    "status": [
        "Success"
    ],
    "buildId": [
        "5875687"
    ],
    "message": [
        ""
    ]
}

LambdaTest runs XCUITest on real devices with one session per test, and it decides the sessions before the run starts. It takes the test list from PatrolGeneratedTests.inc inside RunnerUITests.xctest. Without that file the whole suite runs as a single testInvocations() session.

Enable build-time test discovery

Follow build-time test discovery: set emit_test_manifest: true and switch RunnerUITests.m to the static runner.

Build and upload

#include only compiles the generated tests in. LambdaTest also needs the file itself inside RunnerUITests.xctest, so copy it there after the build:

patrol build ios --release

products=build/ios_integ/Build/Products/Release-iphoneos
cp ios/RunnerUITests/PatrolGeneratedTests.inc \
  "$products/RunnerUITests-Runner.app/PlugIns/RunnerUITests.xctest/"

rm -rf lt
mkdir -p lt/app/Payload lt/tests/Payload
cp -R "$products/Runner.app" lt/app/Payload/
cp -R "$products/RunnerUITests-Runner.app" lt/tests/Payload/
(cd lt/app && zip -qr ../app.ipa Payload)
(cd lt/tests && zip -qr ../tests.ipa Payload)

The copy breaks the local code signature. That's fine: LambdaTest re-signs the apps before installing them on its devices.

Upload lt/app.ipa and lt/tests.ipa with type=xcuit-ios (see docs).

Run

Pass the generated classes as only-testing.

Without only-testing, LambdaTest also runs every Objective-C method whose name starts with test from SDKs linked into the test runner (for example testBlock or testOverride from the Google Maps SDK). The runner links all app pods through inherit! :complete, which Patrol requires. These methods aren't tests, so they fail and mark the build as failed.

It's only strictly needed when such methods exist, but it's safe to always pass it: the list is exactly your Patrol tests. To check whether your app is affected (no output means it isn't):

nm "$products/RunnerUITests-Runner.app/PlugIns/RunnerUITests.xctest/RunnerUITests" \
  | grep -E '[-]\[[^ ]+ test' | grep -v PatrolGeneratedTests_

only_testing=$(awk '/^@implementation /{print $2}' ios/RunnerUITests/PatrolGeneratedTests.inc \
  | jq -R . | jq -sc .)

curl -u "$LT_USERNAME:$LT_ACCESS_KEY" \
  -X POST https://mobile-api.lambdatest.com/framework/v1/xcui/build \
  -H "Content-Type: application/json" \
  -d "{
    \"app\": \"$APP_ID\",
    \"testSuite\": \"$TEST_SUITE_ID\",
    \"device\": [\"iPhone 16-18\"],
    \"only-testing\": $only_testing,
    \"build\": \"Patrol iOS\"
  }"

Each Dart test gets its own session, named after the test.