Blog Infos
Author
Published
Topics
Author
Published
Requirements for Part 3
  • Produce XML & HTML reports for code coverage of on-device tests
  • Run on API 28, API 29, and API 30 devices in Firebase Test Lab
  • Support Android applications targeting API 28, API 29, and API 30
  • Supporting Android Test Orchestrator
  • Combining results with off-device unit tests
  • Supporting multi-module applications
  • Integrate with flank
  • Use scripts to run in a CI/CD pipeline
Integrate with flank
gcloud firebase test android run \
--type instrumentation \
--use-orchestrator \
--no-performance-metrics \
--no-record-video \
--app app/build/outputs/apk/debug/app-debug.apk \
--test app/build/outputs/apk/androidTest/debug/app-debug-androidTest.apk \
--device model=Pixel2,version=29,locale=en,orientation=portrait \
--environment-variables coverage=true,coverageFilePath=/sdcard/Download/ \
--directories-to-pull /sdcard/Download

For this we’ll use the Fulladle plugin (the multi-module Gradle plugin for Flank, not to be confused with the single-module plugin named Fladle). We just need to apply the Fullable plugin by adding the plugin to our project-level build.gradle:

plugins {
    id "com.osacky.fulladle" version "0.17.3"
    ...
}

Then defining a fladle block in the project-level build.gradle:

fladle {
serviceAccountCredentials = \
project.layout.projectDirectory.file("flank-gradle-service-account.json")
smartFlankGcsPath = 'gs://flank_data/results/JUnitReport.xml'
localResultsDir = 'flankResults'
useOrchestrator = true
recordVideo = false
performanceMetrics = false
maxTestShards = 1
devices = [
[ "model": "Pixel2", "version": "29" ]
]
debugApk = project.provider { "${rootProject.projectDir}/app/build/outputs/apk/debug/app-debug.apk" }
environmentVariables = [
"clearPackageData": "true",
"coverage": "true",
"coverageFilePath": "/sdcard/Download/"
]
directoriesToPull = [
'/sdcard/Download'
]
}
view raw build.gradle hosted with ❤ by GitHub
./gradlew runFlank
Use scripts to run in a CI/CD pipeline
./gradlew runFlank | tee results.txt

For any given runFlank execution, all our coverage files are still dumped in a single GCS bucket. We can find the gs:// path for that bucket by running.

gcsbucket=$(cat results.txt | grep matrix_id | awk -F/ '{print "gs://" $6 "/" $7}')

which will yield something like

gs://<project-id/<timestamp>

The bucket will have a separate subfolder for each matrix of tests (typically each test apk that was uploaded) which will then contain individual coverage files for each test (because we’re using the Android Test Orchestrator)

gs://<project-id>/<timestamp>/matrix_0/Pixel2-29-en-portrait/artifacts/sdcard/Download/com.something.app.TestFileAlpha#test1.ec
...
gs://<project-id>/<timestamp>/matrix_0/Pixel2-29-en-portrait/artifacts/sdcard/Download/com.something.app.TestFileOmega#test99.ec
gs://<project-id>/<timestamp>/matrix_1/Pixel2-29-en-portrait/artifacts/sdcard/Download/com.something.feature.TestFileBeta#test1.ec
...
gs://<project-id>/<timestamp>/matrix_1/Pixel2-29-en-portrait/artifacts/sdcard/Download/com.something.feature.TestFileGamma#test99.ec

Retrieving all these files by hand is a bit of a chore, but we can automate some of this. Once we have the gs:// path from above, we can easily retrieve the paths for the individual matrices by running

gsutil ls $gcsbucket | grep -v matrix_ids | grep matrix
gs://<project-id>/<timestamp>/matrix_0/
gs://<project-id>/<timestamp>/matrix_1/
gs://<project-id>/<timestamp>/matrix_2/
gs://<project-id>/<timestamp>/matrix_3/

Job Offers

Job Offers


    Senior Android Software Engineer (f/m/d)

    Paradox Cat GmbH
    Munich
    • Full Time
    apply now

    Mobile Engineer

    OLX Group
    Remote, Portugal, Spain, Romania, Poland
    • Full Time
    apply now

    Android Software Engineer (f/m/d)

    Paradox Cat GmbH
    Munich
    • Full Time
    apply now

OUR VIDEO RECOMMENDATION

, ,

From Scoped Storage to Photo Picker: Everything to know about Storage

Persistence is a core element of every mobile app. Android provides different APIs to access or expose files with different tradeoffs.
Watch Video

From Scoped Storage to Photo Picker: Everything to know about Storage

Yacine Rezgui
Android developer advocate
Google

From Scoped Storage to Photo Picker: Everything to know about Storage

Yacine Rezgui
Android developer ad ...
Google

From Scoped Storage to Photo Picker: Everything to know about Storage

Yacine Rezgui
Android developer advocat ...
Google

Jobs

We can turn around and use gsutil on each one of these to print out the full list of gs:// paths to each coverage file:

gsutil ls $gcsbucket | grep -v matrix_ids | grep matrix | while read -r line; do gsutil ls ${line}Pixel2-29-en-portrait/artifacts/sdcard/Download; done

And then finally use gsutil on each resulting line to download the corresponding code coverage file to app/build/outputs/code_coverageso that we can use our previous techniques to generate the coverage reports that we want.

gsutil ls $gcsbucket | grep -v matrix_ids | grep matrix | while read -r line; do gsutil ls ${line}Pixel2-29-en-portrait/artifacts/sdcard/Download; done | while read -r line; do gsutil cp $line app/build/outputs/code_coverage; done
./gradlew runFlank 
  | tee results.txt
gcsbucket=
  $(cat results.txt 
    | grep matrix_id 
    | awk -F/ '{print "gs://" $6 "/" $7}')
gsutil ls $gcsbucket 
  | grep -v matrix_ids 
  | grep matrix 
  | while read -r line; do gsutil ls ${line}Pixel2-29-en-portrait/artifacts/sdcard/Download; done 
  | while read -r line; do gsutil cp $line app/build/outputs/code_coverage; done
./gradlew jacocoUnifiedReport
TL;DR
  1. In the project-level build.gradle, include the fulladle plugin and add a fladle block.
  2. For our script to automate running the tests on Firebase Test Lab and generate a report, we’ll first execute ./gradlew runFlank and capture the results.
  3. Next we’ll use grep and awk to figure out the gs:// path for the GCS bucket.
  4. Next we’ll use gsutil ls to list the contents of that bucket and then use grepand while read to filter down to just the gs:// paths for each individual matrix’s sdcard/Download folder, and finally use gsutil cp to download all the created coverage file to app/build/outputs/code_coverage.
  5. Finally we’ll run ./gradlew jacocoUnifiedReport to generate an HTML report for all the tests.

This article was originally published on proandroiddev.com on March 07, 2022

YOU MAY BE INTERESTED IN

YOU MAY BE INTERESTED IN

blog
This continuing series of articles will explain how to generate a code coverage report…
READ MORE

Leave a Reply

Your email address will not be published. Required fields are marked *

Fill out this field
Fill out this field
Please enter a valid email address.

Menu