Continuous integration

To run the integration tests on Android, you need 2 apps:

• the normal apk (often called the "app under test"). To build it, run:

  Copy

  ./gradlew :app:assembleDebug -Ptarget="$(pwd)/../integration_test/example_test.dart"
  

• the intrumentation apk. To build it, run:

  Copy

  ./gradlew :app:assembleDebugAndroidTest
  

You need to run the above commands from the android directory of your Flutter project.

Once you have built the apks, use the gcloud tool to run them on Firebase Test Lab:

gcloud firebase test android run --type instrumentation \
    --app build/app/outputs/apk/debug/app-debug.apk \
    --test build/app/outputs/apk/androidTest/debug/app-debug-androidTest.apk \
    --timeout 1m \
    --results-bucket=<RESULTS_BUCKET> \
    --results-dir=<RESULTS_DIRECTORY>

It can be convenient to write a utility shell script to automate this process instead of having to enter these long commands every time you want to run tests. In such case, you might want to take a look at Patrol example app's run_android_testlab script.

First, build your Flutter app, choosing the integration test file as target:

flutter build ios --target integration_test/example_test.dart --debug # or --release

Then, build the UI testing target by running xcodebuild in the ios directory:

xcodebuild build-for-testing \
  -workspace Runner.xcworkspace \
  -scheme Runner \
  -xcconfig Flutter/Release.xcconfig \
  -configuration Release \
  -derivedDataPath ../build/ios_integ \
  -sdk iphoneos \
  OTHER_SWIFT_FLAGS='$(inherited) -D PATROL_ENABLED'

If you want to test a debug version of your app, replace xcconfig and configuration:

xcodebuild build-for-testing \
  -workspace Runner.xcworkspace \
  -scheme Runner \
  -xcconfig Flutter/Debug.xcconfig \
  -configuration Debug \
  -derivedDataPath ../build/ios_integ \
  -sdk iphoneos \
  OTHER_SWIFT_FLAGS='$(inherited) -D PATROL_ENABLED'

Now, go to your project's root directory, and from there go to the build outputs:

cd build/ios_integ/Build/Products

Create a zip archive named ios_test.zip. It will contain both your app under test (Runner target) and UI test runner (RunnerUITests target).

zip -r "ios_tests.zip" "Release-iphoneos" "Runner_iphoneos15.7-arm64.xctestrun"

And finally, upload the ios_test.zip to Firebase Test Lab for execution:

exec gcloud firebase test ios run \
  --test "build/ios_integ/Build/Products/ios_tests.zip" \
  --device model=iphone8,version=15.7,locale=en_US,orientation=portrait

If your .xctestrun file has different iOS version in its name than the device you're running on, simply rename the .xctestrun so that the version will match.

It can be convenient to write a utility shell script to automate this process instead of having to enter these long commands every time you want to run tests. In such case, you might want to take a look at Patrol example app's run_ios_testlab script.