Image generated by Kandinsky 2.2

In Android 13 a new feature called Photo Picker appeared. It allows applications to access individual user photos and videos without giving them full access to the photo/video or the entire memory of the device. This is a good feature for limiting an application’s access to a user’s personal data. It was immediately announced that this feature would also work on Android 11 and newer (with support for Mainline updates), and later its support was added to all devices with Android 4.4 and above via Google Play Services.

I immediately asked myself, why hasn’t it been done before? And the answer is simple — Android didn’t allow to implement this, because Google was struggling with fragmentation and working on the possibility of an independent update of Android, it became a reality.

In this article, I share my opinion about Android fragmentation. You will learn what it is, the reasons for its appearance, and why it is no longer so relevant. I’ll also tell you about the ways Google did with this problem in the first 10 years of Android’s existence and why device updates to newer versions of Android will not be as important as before.

The article is my own opinion and it can be wrong, but I open to discuss it in comments

Brief History of Android OS

To understand the problems that OS developers faced, we need to go back in time. The history of Android begins in 2004. Andy Rubin decided to develop an operating system for digital cameras, which were very popular at the time. However, due to funding problems and the pretty small market for digital cameras, the system was refocused on smartphones as a competitor to Symbian and Windows Mobile.

In 2005, the company was acquired by Google. The Good Corporation decided to enter the mobile phone market by selling its OS to device manufacturers. In 2007, the Open Handset Alliance was created, and at the same time, Android was announced as an open-source mobile platform.

The prototypes of Android devices were similar to Blackberry smartphones: no touch screen and a physical QWERTY keyboard. In short, Blackberry. The year 2007 came, which was marked by the release of the first iPhone. The smartphone has changed the market for smartphones and computers and still sets trends. Google participated in the preparation of the software for Apple’s first smartphone and soon realized which way the wind was blowing, so in a hurry, they began rebuilding Android.

There has been a dramatic change in the concept of an Android smartphone: a touch screen and a minimum of physical buttons. The first Android smartphone, the HTC Dream, was a hybrid between the original Google device vision and the trend set by Apple. The full keyboard was hidden under a large (by the standards of that time) touch screen.

The task for the first Android OS release was simple — to start on the device and to work somehow. Due to limited resources, developers decided to use already existing, free, and popular technologies: Linux kernel, Java, OpenGL, FreeType, Skia. The Linux kernel wasn’t taken in its original form but modified with the features of mobile devices with a small amount of RAM and slow CPUs; they also developed their own Java Virtual Machine, Dalvik, which was different from the standard JVM, taking into account the small amount of RAM, one processor core and device battery use.

Keep in mind that, unlike iOS, from the very first version of Android, a toolkit for developers was open, and the online application store Android Market (now known as Google Play) was launched on October 22, 2008 (4 months after the iOS App Store).

This whole complex and difficult path has affected the features of the Android system and its architecture. I think this often happens to developers: we create MVP without planning for scaling, then new requirements come and the deadlines are running out… It’s one thing to have this kind of problem when you are developing an application. But it’s a completely different level when you are building an Operating System that cannot be updated so easily.

Android 1.0 Architecture

It is also necessary to understand that Google only developed the OS, meanwhile vendors took the system, modified it and updated it independently of the corporation. Developing the Android OS, Google focused on devices with a touch screen that support modern technologies. It was necessary to increase the speed of the OS and give more features to developers and users.

Fragmentation

Fragmentation is a problem due to the fact that the market has a huge number of devices running on different versions of the operating system and with different modifications from manufacturers. Because of it, the fact that the application works on an emulator or on one device didn’t and still doesn’t give a guarantee that it will work correctly on all devices.

This problem leads to the fact that developers need to spend more time developing, adapting, and testing applications, and they do this only for the most popular devices among the users.

Device Fragmentation in Android. Source

You can see the visualization of the fragmentation of Android devices in 2014, and today the picture will only get worse. The adaptation of new versions of the OS also leaves much to be desired, but before, it was worse.

On the one hand, fragmentation is a pain, but on the other, it is a huge opportunity and space for innovation among device manufacturers. They can implement their own features, create a unique interface, and stand out from the others. Thanks to this, we saw folding smartphones, gaming ones, with E-ink display, sliders, modular and other formats.

If you look at smartphones on Windows Mobile or Windows Phone, they were all the same in software and appearance. Just because Microsoft didn’t allow vendors to modify the OS. Nokia had this right, but they decided not to use it so the user experience that Microsoft implemented wouldn’t be disrupted. Which one of these approaches is good and which one is bad is not for me to judge, but the time has reasoned — the Android OS is alive and well, being the most popular OS not only among mobile Operating Systems but even among all OSes. At the same time, all developments for mobile versions of Windows were closed in 2017, and the support was ended in 2020.

Vendors Updates

Fragmentation is also caused by the fact that each manufacturer updates their devices independently, which is also a problem. Additionally, companies don’t always update their new devices to the latest version of Android because it can interfere with their main goal (sell as many devices as soon as possible).

Android device manufacturers couldn’t update their devices quickly because the update cycle requires a few steps: company needs to get the source in AOSP and the latest drivers for the microchips in the device, then merge all the changes with company’s shell; after that, company needs to adapt the changes from the new versions of the OS into their shell, and then send it all to the user devices.

This process takes time; the queue of models for update is completed according to the principle of «newness of the device and its price». Budget models weren’t updated at all because the manufacturers didn’t provide support for them. There were cases when vendors sent dummy patches; updates actually arrived, but there was nothing in them. Such updates can be called placebos because some users believe that they make their devices faster and safer.

It seems that Google should have done something at this point, but either they couldn’t do it or they were afraid to put pressure on device manufacturers at the dawn of Android. In some cases, Google had to intervene, though. For example, they had a showdown with HTC because, at the start, they didn’t update their devices at all, regardless of their price.

Android Jetpack

The development of new versions of the Android SDK and the slow update of devices led to the fact that the market was flooded with devices with all possible versions of Android. It was important for the business that all these devices were supported, so the developers were set a task. The code was flooded with ‘if…else’ statements that checked the version of Android on the device.

if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.LOLLIPOP) {
 // CODE for Android 5.0 Lollipop and higher
} else if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.ICE_CREAM_SANDWICH) {
 // CODE for Android 4.X
} else if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.HONEYCOMB) {
 // CODE for Android 3.X
} else {
 // CODE for all other Android versions supported by the application
 // (minSdk from AndroidManifest and higher)
}

Google decided to do this all by themselves for the most popular APIs from the Android SDK, introducing the Android Support Library, and later reformed it into Android Jetpack, also called AndroidX.

It contained many different Compat APIs that allowed applications to call APIs from newer versions of Android on older ones. On older versions of the OS, this often led to the call being simply ignored, although for some API calls were resent to analogs from the older version of the Android SDK.

Jake Wharton

The next big step was to port the new APIs, which had no analogs in the older versions of the SDK. The community started doing this earlier; for example, one of the most popular libraries that made Jake Wharton a legend — ActionBarSherlock. The library was the first version of AppCompat and ported the Action Bar from Android 3.0 to Android 2.X

Motorola Xoom is the first Android Honeycomb tablet

Why did an enthusiast do that and not Google? The release of Android 3.0 changed Android very much, because it was the first OS with official support for tablets and also the first time in history that Google didn’t open the source code of Android.

This happened due to the fear of porting the latest tablet version of Android to smartphones. In Honeycomb, the first version of Holo was introduced — the branded Android design, which took hold until 2014. When Android 5.0 and Material design, still relevant today, were introduced, its third version, Material You, was also released.

Google Play Services for distributing functionality

Android Open Source Project (AOSP) is an empty version of Android OS where you will not find many of your usual applications and services.

What you will find on devices from Samsung, Google, Xiaomi, and other popular Android smartphone manufacturers is a Google-certified version of Android with Google Mobile Services: Google Play application store, Google search, Chrome browser, Google Play Services, as well as other default Google software.

Google Play Services is a framework that allows developers to use various features and services from Google in their applications. This framework includes features such as Google authorization, location, Firebase services, Push messages, analytics, and other services that developers can use to improve their applications. Without Google Play Services, you won’t be able to run Google WearOS, which, through proprietary services, provides data transfer between devices as well as some features from the Android SDK.

Google Play services receive regular updates that contain improvements and bug fixes.

The company’s branded services allow you to distribute updates or provide functionality across all OS versions. For example, if you are learning how to get a user’s location, then in the official documentation you will find a guide that is based on the Fused Location Provider from Google Play Services rather than the location API from the Android SDK.

Google Play Services are installed on most Android devices and are updated automatically via the Google Play Store. All updates in services are delivered regardless of the device manufacturer, operator, and Android version. This is an important point since Google can deliver bug fixes, new features, and other improvements on their own, unlike firmware and libraries plugged in by application developers. This is also a way to fight fragmentation when the company isn’t tied to device manufacturers and can control the API update process independently. It is a working solution, but it requires developers to connect libraries to the client API to interact with the services from Google Play Services.

It looks like everything is fine now, but that’s not true. With time, Google services have also become evil. To install Google Play and services of the same name, any vendor must comply with Google’s strict rules, which include pre-installation of the Google Play application, a folder with Google’s applications on the main screen of the default launcher, as well as many other hidden terms and special conditions for vendors. Why is everyone doing this? Google Play has the largest catalog of Android applications, counting the top 1000 most popular Android applications. All guidelines from Google are inspired by the use of these services, and all developers aim to work with Google services, even if their software is distributed through other application stores. This is due to the fact that Google services are available on almost any popular device outside of China. Google Services are king and god on Android devices, and developers can only create features based on them. Don’t think that if another company had the opportunity to make their services with the same privileges, they would not use it. It is the goal of any developer, both firmware and third-party, to have a guarantee that their applications will work on the device.

You can say: “But we have AOSP! Why won’t vendors use it? Isn’t that an open version of Android?”. First, it has its own license, which also needs to be studied. Throughout the documentation, you will find the phrase Open Source Code, which doesn’t mean general availability. Have you ever used an AOSP device? Naked Android on the Nexus and Pixel isn’t AOSP. AOSP is an Android build without an application store; without normal standard applications for working with email and SMS, there will be no push notifications, and some of the features advertised by Google will simply be missing. So that leaves vendors with two options: accept the terms of Google or create all the services themselves and hope that people will buy the device and developers will adapt the software. Huawei and vendor devices in China were forced to take the second path since there have never been Google Play Services in China.

For ordinary users, Google Services have become an integral part of the Android OS experience, and Google gains control over all devices that they don’t even produce, although it isn’t complete. It makes sense. After all, the company spends huge sums on the development and support of the operating system, whereas other companies do almost all the developments only for themselves.

A/B System Updates

A/B updates are a feature that allows you to update your device without interrupting applications or affecting user data. They work by downloading an additional system image, which is installed on an inactive partition, and then switching to the active partition after rebooting the device. It allows the device to quickly switch between different versions of the system without the need for a full reboot.

You may think: “What does it have to do with the fragmentation?”. The thing is: the new system has allowed the user to make transparent system updates. These updates don’t force the user to wait for updates during the boot of the device. Instead, an imperceptible reboot that the system can perform while the user isn’t using the device takes place. This is clearly noticeable when your device requests an unlock code password and a fingerprint or facial scanner aren’t allowed to be used. The update installation takes place while the device is running in the background. It’s still slow, but it doesn’t matter if you can use your favorite apps and play games.

Direct Boot

If a device reboot can take place unnoticeably for the user, how do the applications continue to work after this quiet reboot? For this, Google created a Direct Boot mechanism that allows applications to be launched before the user enters the device password. Applications are able to use a part of the storage that doesn’t require the user to unlock the device; it’s enough to just boot the smartphone. This way, you can easily receive SMS, phone calls, alarms, push notifications, etc. This feature allows critical functionality to remain operational and prevents the user from suspecting that the device was rebooted. It was a great move from Google, as it prepared Android for future changes to decouple updates from device manufacturers.

Project Treble (Android 8)

The Android update process was slow until Android 9. The reason is that device manufacturers had to get driver updates for all the microchips in the device, source code for the new OS, and then adapt their shell and other developments. Google has decided to break the dependencies between these parts.

Android Release process before Treble. Source

Project Treble is a major change in the Android architecture, implemented in Android 8 Oreo, which divides the OS into two parts: the Android system and low-level software for running the device (they are also called vendor implementation). Between them, a new layer was added, the vendor interface, something similar to the interface in the code. Device developers are provided with a layer that guarantees backward compatibility and passes special tests — the Vendor Test Suite (VTS).

Device update before Treble & with Treble. Source

Thanks to Project Treble Generic System Images (GSI) appeared, which is compiled from Android sources and guarantees backward compatibility with the 3 previous versions of the vendor’s implementation. This means that GSI Android 13 can be installed on any device running Android 12, 11 and 10.

Project Treble support is obvious for all devices that were released with Android 8 or higher. For those who updated from the previous version, this requirement was optional.

Refactoring of the system architecture has allowed manufacturers to update devices to new versions of Android faster, as they don’t need to wait for driver updates anymore. Instead, they can work with a standardized interface and do their part of the firmware adaptation.

Speed of distribution of new versions of Android

Treble had an effect on the speed of device updates, as can be seen in the graph, and thus had a direct impact on reducing the degree of device fragmentation, but this still didn’t solve problems with older versions of the operating system. They hung like a dead weight, but new versions were eventually successfully replaced over time.

Distribution of versions on Android devices on June 6, 2023

If you look at it today, the versions of Android 8 and below aren’t growing in shares, but the newer versions are only gaining momentum. Android 10 and higher occupy more than two-thirds of the entire market.

Modular system of OS components

Project Treble was a good solution, but it still left Google dependent on vendors and their responsiveness and willingness to update devices. They needed to find a solution to update devices, regardless of manufacturers.

The solution was obvious — to divide the system into parts that could be delivered via the system service on a device with full permissions. That’s how Project Mainline appeared in Android 10, which separated system components and allowed them to be updated independently of the normal operating system update cycle.

Now the question is: who in Android can have enough rights to update the modules? Of course, Google Play Services! But they’re not the only ones who can do it. Any service with the same rights from the device manufacturers is suitable.

Module updates cannot introduce new APIs within them but can change behavior. In the new versions, APIs can expand, but backward compatibility is guaranteed. This behavior is guaranteed by the Compatibility Test Suite (CTS), a special set of tests through which Google checks the implementation of the API by device vendors. All modules are updated or rolled back automatically.

For example, such a system allowed to receive updates on the network stack and time zones without waiting for the vendor to do it or the application developer to upload the new version of the database.

APEX

To distribute the modules in the Android SDK, a new file format, APEX and a utility for its installation during the boot of the device, APEX Manager (apexer) were introduced. The APEX format is similar in structure to the APK, but the APK isn’t suitable because it cannot be installed during the loading phase of the operating system.

An APEX file is a valid APK file because it uses the same signature scheme and contains AndroidManifest, which is required for any APK. All this is necessary to fully use all the tools for APK files: APK installer on the device, adb, signature utility, PackageManager API.

APEX file structure

Android SDK Extensions

In Android 11, the framework developers went further and decided to implement the ability to add new APIs for older versions of Android. In fact, this feature began to extend the Android SDK with new APIs. Updatable modules can now add new features, and Android SDK Extensions add APIs for developers to use these features.

// Check if we have Android 13
if (Build.VERSION.SDK_INT >= Build.VERSION_CODE.TIRAMISU) {
    // Launch PhotoPicker
    context.startActivity(Intent(MediaStore.ACTION_PICK_IMAGES))
}

// The code runs on Android 11 and higher
// Check that the extension is available and that it is the proper version
if (SdkExtensions.getExtensionVersion(Build.VERSION_CODES.R) >= 2) {
    // Launch PhotoPicker
    context.startActivity(Intent(MediaStore.ACTION_PICK_IMAGES))
}

The result was the introduction of the SDK Extensions module in the OS. SDK Extensions have a version something similar to the way we check the API using Build.VERSION_CODE, but thanks to the extensions, now in the documentation you can see that it is not only the API level, where the API is available, that is described. But also the set of SDK Extensions and their versions that will contain this API.

Example of a description of availability in SDK Extensions

When setting up a project, we now need to specify the compileSdk version as well as the version of SDK extensions. Don’t forget to install it first in Android Studio SDK Manager. When using some Android features, the system now checks not the Android SDK level but the version of the extensions. Android Lint will surely remind you about that.

// build.gradle.kts
android {
    compileSdk = 33 // Android 13 T

    // T Extensions will be used
    // Specify the version of the extensions. Not connected by default
    compileSdkExtension = 5
}

Do we really need new Android versions ?

The main question remains, is it worth waiting for a new version of the OS, and when the update will come to your smartphone? I think NO. Each version of Android brings fewer new features for users, and many new APIs for developers are delivered through libraries within Jetpack and Google Play Services. All Google efforts are aimed at having full control over OS updates, regardless of device manufacturers. The latter will not be able to jump off Android so easily; there are simply no alternative OSes. Developing your own is time-consuming and expensive. Google’s managers have done very well, giving a lot of freedom and opportunities to lure manufacturers and developers, but they have been working for years to regain full control to themselves, and, as I told you, they are succeeding.

Every year I review the next release of the Android OS, and I see how many changes are happening under the hood: the release of new Mainline modules, the integration of technologies from Jetpack to the Android-framework level, the integration of new technologies related to hardware. This shows that Google makes Android for themselves, partly for their partners, pursuing their own business interests.

Write in the comments what you think about the development trends of the Android OS. I will be happy to discuss them.

This article was previously published on proandroiddev.com