Recent conversations around Android Runtime Optimizations has to do with a technique known as Profile-Guided Optimization. (or Profile-Directed Feedback) The idea behind it is that the software engineer can consider various code paths a user would typically go through and pre-compile these paths instead of leaving it to the JIT compiler. This is certainly interesting but not without caveats. But before we dive into that, we should do a quick review of how Java works.

What makes Java interoperable across various OS and CPU architectures is because of the JVM. How Java compilers work is that they convert the Java source code into bytecode. The bytecode is then interpreted by the JVM and is converted to machine code depending on the OS and CPU architecture it runs on. Below is how the whole process looks like:

An Android engineer might know by now that Android does not use JVM. Because Android runs on memory and space-constrained devices, Dalvik is used in its stead (and since Lollipop, ART)¹ Below is the modified Android compilation and code execution path:

Unlike Dalvik which depended on JIT (Just in time Compilation), Android Runtime depended largely on AOT (Ahead of Time compilation). How JIT works is by having the Java bytecode converted into machine code at runtime. This allows Java to remain interoperable and also usable, especially in storage-constrained environments.

AOT to the Rescue?

One might ask, “if JIT compiles the code at runtime, wouldn’t it be slow?” The answer is yes. JIT sacrifices performance over storage. (Not entirely true. More on this later) Of course, a counterproposal has to be raised. At a time when storage is now so cheap, can we then optimize for performance over storage? Those who thought such would then flock to Ahead of Time Compilation (AOT). In this strategy, Java pre-compiles various parts of the code so that the JIT doesn’t have to take that responsibility. The drawback of course is that because AOT code has to be executed across various operating systems and CPU architectures, all of these pre-compiled machine code has to come together under the same java package, thereby bloating up the file size.

This is not a problem when the OS is not often updated. However, because Android and its OEMs regularly patch their OS, precompilation has to be done every time there’s an update. This leads to a significant amount of wait time for users who have just updated their phones.

Enter Crowdsourced Data

Now let’s go back to JIT for a second. More modern implementations of JIT comes with an execution counter so that it knows what part of the code path to pre-compile, hence improving performance. This of course is based on how the individual user interacts with the application. Now, suppose the JIT has access to the cloud, such that it could crowdsource the code paths used by various users around the world. Could it, in theory, pre-optimize the execution for the user? The answer is yes. Now that the cloud is widely available, JIT can rely on crowdsourced data on the cloud to make apps run even faster. For Android, Google calls this Cloud Profiles.²

Cloud Profiles, however, has one problem. While they excel at identifying execution paths, they are only as good as the lifespan of the app version. When the app is updated, Cloud Profiles for the previous app version are rendered useless as shown below:

This is because Cloud Profiles won’t have data about how the new app version is executed. Hence, is unable to pre-optimize for early-access users. Of course, this will taper over time, but the problem remains for early adopters, and if the company does weekly deployments, such as Twitter and Netflix, the benefits of Cloud Profiles are diminished.

Profile Guided Optimizations?

Recently, Android introduced Profile Guided Optimizations through the use of Macrobenchmarking and Baseline Profiles. This is to solve the problem mentioned above. In essence, the idea is to find the right balance between JIT and AOT so that only the code paths the software engineer thinks users often go to are optimized. If the AOT codes come shipped with the app, then one can find the right balance between having just the right AAB file size and also having the app run as smoothly as it can be. But there’s one problem.

Consider Grab, which surfaces various home screens depending on which country you’re in:

Here, Profile Guided Optimizations lose their appeal. Because now, shipping profiles tied to each country tends to be sub-optimal. Moreover, when the use case branches out into various features users might use in a super app, the value of Profile Guided Optimizations quickly diminishes.

In fact, this 2019 talk noted that AOT tends to be almost always suboptimal compared to JIT because it relies on highly specific code paths which may not always be true in actual usage.

Given enough time, JIT will always outperform AOT since it has a larger pool of data to use. Moreover, one does not have to predict the range of code paths users will go through prior to them actually going through them.

What’s Next?

In the end, all is not lost. Because most Android apps come shipped with Firebase and run on devices with Google Play Services, these two can coalesce and send common usage data to the cloud, as they already do with Firebase Analytics, Firebase Test Lab, and Android Vitals. A user’s typical usage patterns won’t vary across app versions, assuming that the means to do so remain intact, the data from these services can help perform automated profile-guided optimizations based on large volumes of data. Hence, further optimizations for app performance are likely to happen on the Google Play side of things where Cloud Profiles will attempt to pre-optimize areas of optimal usage that are personalized in nature. This of course is a storage hog.³ Yet now that storage is significantly cheaper than it used to be, incrementally personalizing this runtime optimization is certainly not far-fetched.

In conclusion, one is advised to be mindful of using Baseline Profiles even when the recent Android Dev Summit and various online resources push for this. For Southeast Asia where super apps are quite common, this can come at a significant human capital expense with little to no benefit. If Google Maps increased searches by only 2.4%, and it’s one of the top of the funnel activities, improvement on essential conversion metrics, such as purchasing a product, is even much lower. So if it’s not worthwhile to spend hundreds or even thousands of dollars of man-hours for baseline profiles, I recommend holding back. Cloud Profiles will be that which would save the day, and you barely have to lift a finger.

___

¹ Technically you can turn on ART on KitKat via Developer Options but it didn’t become the default option until Lollipop.

² Of course, the Java world has various alternatives such as JITaaS.

³ But Google just gave everyone a free 1 TB storage so there’s that.