DALL·E generated image (shakespeare searching for something in vaporwave world)

NOTE: While not being an actual continuation of my previous medium post, it’ll be great if you read that as well to get an info on other issues that you might face while measuring the impact of Baseline Profiles on OPPO, OnePlus or Xiaomi devices.
Also as there are already many articles and documentation about different app startup states and Baseline Profiles I’m not planning to cover it here, instead if you’re curious you can scroll to the bottom of this post to check the reference links.

Recently I was on a mission to improve App Cold Start performance for one of my client. I was able to generate the Baseline Profiles without any issues as I have done similar exercises a few times already in the past. I check the impact of it on my iQOO 7 (specs: SD870G + 8GB RAM + 128GB & Android 13) device and I was able to see a decent improvement where the time spent by Jit thread pool during app cold start got reduced to 41ms, which was 212ms earlier i.e. 80% reduction 🙂.

Then I was very eager to do the same exercise on a budget device OPPO A15 (specs: Helio P35 + 2GB RAM + 32GB & Android 10) as here the time spent by Jit thread pool was 2180ms (i.e. ~2.2s) so I was expecting to save atleast 1000ms (i.e. 1s). So I did the benchmarking with a baseline profile build and to my surprise, I saw no reduction in app cold start time 🤔. So I opened one of the systrace generated during the profiling using macrobenchmark and checked the time spent by Jit thread pool which was still very similar i.e. ~2048ms.

My first thought was that maybe there was an issue in the test execution so I ran the tests 2 more times but still there was no improvement. Then I thought maybe because Android 10 it’s not working properly, so I checked the documentation and found that it should work on all Android API24+ devices. Finally, I came to the conclusion that it was time to put the Sherlock hat and get the magnifying glass ready to investigate the issue. 🕵

Note: I used AGP version 7.3.1, MacroBenchmark version 1.1.1 & ProfileIntsaller bersion 1.2.2 for measuring Baseline Profile impact.

Investigation

As I can’t show the actual project that I’m working on for many reasons I’m going to use Now in Android App to demonstrate the same issue. I ran Cold Start tests with both with and without Baseline Profile on the same Oppo A15 device. As you can see in the below screenshots the time spent by Jit thread pool with and without Baseline Profiles was nearly same i.e. 367ms and 337ms respectively.

Jit thread pool spending 367ms on Oppo A15 device on the tests w/o Baseline Profiles

Jit thread pool spending 337ms on Oppo A15 device even on the tests with Baseline Profiles

For reference you can also see the result of the same tests on iQOO 7 device below where the time spent by Jit thread pool with and without Baseline Profiles was 75ms and 0.6ms 😲 respectively.

Jit thread pool spending 75ms on iQOO 7 device on the tests w/o Baseline Profiles

Jit thread pool spending ~0.1ms on iQOO 7 device on the tests with Baseline Profiles

Then I started digging into the logs generated during the BaselineProfile tests to see if I can find some clue. After spending few hours I found few statements (same count as the test iteration count) saying “Skipping profile installation for com.google.samples.apps.nowinandroid.benchmark” as shown below, which made it concrete that for some reason Baseline Profiles were not being installed properly.

Logs indicating that profile installation was skipped for some reason on tests with Baseline Profiles

Then I checked all the known issues with Baseline Profiles and finalized two of the points mentioned there that can interfere with Baseline profile installation.

• Battery optimizations may interfere with profile installation. To ensure that your profiles are installed effectively, disable any battery optimizations on your benchmark devices.
• Android Studio Profilers don’t install Baseline Profiles when profiling the app (issue).

I verified that there was no battery optimization enabled on Oppo A15 and battery was also at 100%. About the second issue mentioned above, from the good UX perspective, I thought I should’ve gotten an error if it was the case, similar to the ones we get when doing profiling on device with low battery or an emulator. Finally, I thought about trying latest releases of macrobenchmark (i.e. 1.2.0-alpha09) and profileinstaller (1.3.0-alpha03) to verify that it’s not an issue with the dependencies that I was using.

I bumped the versions and then ran the tests again but this time the test threw an error saying “Check failed”. This at least made me happy as now I know that there is an issue with baseline profile tests, unlike earlier where if the User doesn’t check the logs then they don’t get to know why the baseline profiles were not working. Then I followed the stacktrace, which led me to cmdPackageCompile() function where the result of “cmd package compile -f -m $compileArgument $packageName” was being matched to “Success” string as shown below.

Tests with Baseline Profiles failing at CompilationMode.cmdPackageCompile()

This made me wonder why I this error didn’t happen earlier so I reverted the version bump and checked the same cmdPackageCompile() function and saw that in earlier versions the result of the same package compile command was not being asserted 💡, which made it clear why there was no error earlier. For reference you can see the source code of cmdPackageCompile() function till androidx.benchmark:benchmark-*:1.2.0-alpha06 & above in below screenshots.

CompilationMode.cmdPackageCompile() source code till androidx.benchmark:benchmark-*:1.2.0-alpha06

CompilationMode.cmdPackageCompile() source code for androidx.benchmark:benchmark-*:1.2.0-alpha07+

"adb shell cmd package compile -f -m speed-profile $YOUR_PACKAGE_NAME"

This article was originally published on proandroiddev.com