Blog Infos
Author
Published
Topics
Author
Published
Requirements for Part 3
  • Produce XML & HTML reports for code coverage of on-device tests
  • Run on API 28, API 29, and API 30 devices in Firebase Test Lab
  • Support Android applications targeting API 28, API 29, and API 30
  • Supporting Android Test Orchestrator
  • Combining results with off-device unit tests
  • Supporting multi-module applications
  • Integrate with flank
  • Use scripts to run in a CI/CD pipeline
Integrate with flank
gcloud firebase test android run \
--type instrumentation \
--use-orchestrator \
--no-performance-metrics \
--no-record-video \
--app app/build/outputs/apk/debug/app-debug.apk \
--test app/build/outputs/apk/androidTest/debug/app-debug-androidTest.apk \
--device model=Pixel2,version=29,locale=en,orientation=portrait \
--environment-variables coverage=true,coverageFilePath=/sdcard/Download/ \
--directories-to-pull /sdcard/Download

For this we’ll use the Fulladle plugin (the multi-module Gradle plugin for Flank, not to be confused with the single-module plugin named Fladle). We just need to apply the Fullable plugin by adding the plugin to our project-level build.gradle:

plugins {
    id "com.osacky.fulladle" version "0.17.3"
    ...
}

Then defining a fladle block in the project-level build.gradle:

fladle {
serviceAccountCredentials = \
project.layout.projectDirectory.file("flank-gradle-service-account.json")
smartFlankGcsPath = 'gs://flank_data/results/JUnitReport.xml'
localResultsDir = 'flankResults'
useOrchestrator = true
recordVideo = false
performanceMetrics = false
maxTestShards = 1
devices = [
[ "model": "Pixel2", "version": "29" ]
]
debugApk = project.provider { "${rootProject.projectDir}/app/build/outputs/apk/debug/app-debug.apk" }
environmentVariables = [
"clearPackageData": "true",
"coverage": "true",
"coverageFilePath": "/sdcard/Download/"
]
directoriesToPull = [
'/sdcard/Download'
]
}
view raw build.gradle hosted with ❤ by GitHub
./gradlew runFlank
Use scripts to run in a CI/CD pipeline
./gradlew runFlank | tee results.txt

For any given runFlank execution, all our coverage files are still dumped in a single GCS bucket. We can find the gs:// path for that bucket by running.

gcsbucket=$(cat results.txt | grep matrix_id | awk -F/ '{print "gs://" $6 "/" $7}')

which will yield something like

gs://<project-id/<timestamp>

The bucket will have a separate subfolder for each matrix of tests (typically each test apk that was uploaded) which will then contain individual coverage files for each test (because we’re using the Android Test Orchestrator)

gs://<project-id>/<timestamp>/matrix_0/Pixel2-29-en-portrait/artifacts/sdcard/Download/com.something.app.TestFileAlpha#test1.ec
...
gs://<project-id>/<timestamp>/matrix_0/Pixel2-29-en-portrait/artifacts/sdcard/Download/com.something.app.TestFileOmega#test99.ec
gs://<project-id>/<timestamp>/matrix_1/Pixel2-29-en-portrait/artifacts/sdcard/Download/com.something.feature.TestFileBeta#test1.ec
...
gs://<project-id>/<timestamp>/matrix_1/Pixel2-29-en-portrait/artifacts/sdcard/Download/com.something.feature.TestFileGamma#test99.ec

Retrieving all these files by hand is a bit of a chore, but we can automate some of this. Once we have the gs:// path from above, we can easily retrieve the paths for the individual matrices by running

gsutil ls $gcsbucket | grep -v matrix_ids | grep matrix
gs://<project-id>/<timestamp>/matrix_0/
gs://<project-id>/<timestamp>/matrix_1/
gs://<project-id>/<timestamp>/matrix_2/
gs://<project-id>/<timestamp>/matrix_3/

Job Offers

Job Offers


    Kotlin Multiplatform Mobile Developer

    Touchlab
    Remote
    • Full Time
    apply now

    Kotlin Backend Developer and Mobile Enthusiast (m/f/d)

    Axel Springer National Media & Tech
    Berlin
    • Full Time
    apply now

    Senior Android Engineer – Outfits Program

    Zalando SE
    Berlin
    • Full Time
    apply now
Load more listings

OUR VIDEO RECOMMENDATION

, ,

The Evolution of Android Graphics in Android 12/13

Android 12 and 13 both added significant new capabilities to Android platform graphics, including RenderEffect, RuntimeShader, and more. At the same time, RenderScript has been deprecated and we’ve introduced the RenderScript Intrinsics Replacement Toolkit. This…
Watch Video

The Evolution of Android Graphics in Android 12/13

Daniel Galpin
Android Developer Advocate and Fast Talking YouTuber
Google

The Evolution of Android Graphics in Android 12/13

Daniel Galpin
Android Developer Ad ...
Google

The Evolution of Android Graphics in Android 12/13

Daniel Galpin
Android Developer Advocat ...
Google

Jobs

We can turn around and use gsutil on each one of these to print out the full list of gs:// paths to each coverage file:

gsutil ls $gcsbucket | grep -v matrix_ids | grep matrix | while read -r line; do gsutil ls ${line}Pixel2-29-en-portrait/artifacts/sdcard/Download; done

And then finally use gsutil on each resulting line to download the corresponding code coverage file to app/build/outputs/code_coverageso that we can use our previous techniques to generate the coverage reports that we want.

gsutil ls $gcsbucket | grep -v matrix_ids | grep matrix | while read -r line; do gsutil ls ${line}Pixel2-29-en-portrait/artifacts/sdcard/Download; done | while read -r line; do gsutil cp $line app/build/outputs/code_coverage; done
./gradlew runFlank 
  | tee results.txt
gcsbucket=
  $(cat results.txt 
    | grep matrix_id 
    | awk -F/ '{print "gs://" $6 "/" $7}')
gsutil ls $gcsbucket 
  | grep -v matrix_ids 
  | grep matrix 
  | while read -r line; do gsutil ls ${line}Pixel2-29-en-portrait/artifacts/sdcard/Download; done 
  | while read -r line; do gsutil cp $line app/build/outputs/code_coverage; done
./gradlew jacocoUnifiedReport
TL;DR
  1. In the project-level build.gradle, include the fulladle plugin and add a fladle block.
  2. For our script to automate running the tests on Firebase Test Lab and generate a report, we’ll first execute ./gradlew runFlank and capture the results.
  3. Next we’ll use grep and awk to figure out the gs:// path for the GCS bucket.
  4. Next we’ll use gsutil ls to list the contents of that bucket and then use grepand while read to filter down to just the gs:// paths for each individual matrix’s sdcard/Download folder, and finally use gsutil cp to download all the created coverage file to app/build/outputs/code_coverage.
  5. Finally we’ll run ./gradlew jacocoUnifiedReport to generate an HTML report for all the tests.

This article was originally published on proandroiddev.com on March 07, 2022

YOU MAY BE INTERESTED IN

YOU MAY BE INTERESTED IN

blog
This continuing series of articles will explain how to generate a code coverage report…
READ MORE

Leave a Reply

Your email address will not be published.

Fill out this field
Fill out this field
Please enter a valid email address.

Menu