Google has been working on reducing fragmentation on Android for years, though part of the cause of that is the inherent nature of Android and the dual-edged sword of choice and freedom. There are countless OEMs active in the space, and all of them want to make their own modifications for their own devices. The problem then is that it looks like Android OS updates are slow to roll out across the board, but there’s not a lot that Google can actually do to force OEMs to update their devices. As such, the next best thing that Google can do is make the update process as easy and frictionless as possible.
Easing the Android update pain
The first major initiative in Google’s long-term project to reduce the development burden was Project Treble. Announced alongside Android 8.0 Oreo in 2017, Project Treble modularized Android by separating the OS framework from the vendor implementation (HALs and the device-specific Linux kernel fork). This made it easier for Android OEMs to rebase their OSes on top of the latest AOSP framework, as they could boot the latest version without needing updated code from vendors. As a result, OEMs could ready their custom Android forks quicker than before, and by extension, roll out major OS updates more quickly.
The next step in Google’s plans was to streamline the delivery of updates to key Android components. Google called this initiative Project Mainline when it introduced it alongside Android 10 in 2019. Google essentially took control of key OS components and forbid OEMs from modifying them. They then set up a delivery mechanism via Google Play so they could remotely roll out updates to these key components without having to wait for OEMs to apply the patches themselves. Mainline greatly improved how quickly devices receive updated versions of important OS components, in turn improving the security of the Android ecosystem as a whole.
When it comes to Treble though, the Linux kernel realistically shouldn’t be lumped in with closed-source vendor code. Todd Kjos at this year’s Linux Plumbers Conference has explained in the past the difficulties that are faced when it comes to fragmentation on Android, and a lot of it now centers around the Linux kernel that OEMs ship with their devices. For context, Google forks each mainline Linux kernel into an “Android Common Kernel” (ACK) branch, which closely tracks the mainline release but adds a few Android-specific patches. SoC vendors like Qualcomm, MediaTek, and Samsung then fork that kernel for each SoC they make. OEMs then take that SoC-specific kernel and add additional patches to implement support for the specific hardware they want to ship.
The above diagram shows how a device’s kernel goes through several layers of change that abstract it far from the Linux LTS kernel. To simplify it, we start with the Linux Kernel, and it gets merged into the Android Common Kernel with a few changes. From there, the Android Common Kernel gets merged into a vendor kernel (Qualcomm, MediaTek, etc) with its own modifications and changes. Finally, the vendor kernel is merged into an OEM’s device-specific kernel. By this stage, any one device’s kernel is far removed from the Linux LTS kernel that it started with.
As a result of all of those forks, as much as 50% of the code running on an Android device is out-of-tree code, which means that it’s not from upstream Linux or AOSP common kernels. This makes it incredibly difficult (not to mention time-consuming and costly) to merge upstream changes. For OEMs, there’s no incentive to do so, but that practice can be harmful to device security. This is also why a lot of Android devices are left on older LTS kernel releases, which has the side effect of devices losing out on access to new Linux kernel features.
Android is fragmented, and Google knows it
Google knows full well that this is a problem, and even has a section called “The costs of fragmentation” in the Android developer documentation. Google says that “most flagship devices ship with a kernel version that’s already at least 18 months old”. Even worse, Google also says that “Android 10 supports 3.18, 4.4, 4.9, 4.14, and 4.19 kernels, which in some cases haven’t been enhanced with new features since Android 8 in 2017.” This makes it difficult to add features that require new Linux kernel versions. Linux kernel 3.18 was launched in December 2014, back when Android 5.0 Lollipop was the latest version of Android. That’s clearly a problem and can hold the platform back.
For example, Code Aurora Forum, or CAF for short, hosts the source code for various Qualcomm Snapdragon SoCs. Qualcomm, as an SoC vendor, distributes a forked version of the Linux kernel to OEMs/ODMs, and those companies then add device-specific changes on shipping devices. This is what adds several layers of fragmentation. In addition, Qualcomm makes changes to the AOSP framework to optimize Android for each of the company’s Snapdragon mobile platforms. Qualcomm privately distributes its modified Linux kernel, AOSP framework, and other software tools to its partners as part of a Board Support Package, or BSP. CAF is where Qualcomm publically publishes these Linux kernel changes and AOSP framework changes.
This CAF release can be useful for custom ROM developers who wish to use it as a starting point rather than pure AOSP, which is why you sometimes see “CAF-based” ROMs on our forums. Remember the Snapdragon 625 that seemed to power so many mid-range smartphones for years? That launched with Linux Kernel 3.18, and only towards the end of 2018 (two years after the chipset launched) did Qualcomm update the kernel sources and publish them on CAF for msm8953 (the chipset name of the Snapdragon 625) bringing support for Linux Kernel 4.9. The problem is that most OEMs won’t update phones to this new Linux kernel version, especially not mid-range phones two years after the chip was released. Admittedly, it’s very rare for a major kernel update like that to even happen in the first place, but the point is that it has happened, so it’s not just an impossible scenario.
All in all, the current fragmentation in Android is a mess, to put it lightly. Google’s latest attempts to fix that fragmentation come in the form of the Generic Kernel Image, or the GKI.
Introducing the Generic Kernel Image
In order to address this fragmentation, Google worked on the Android Generic Kernel Image (GKI). This is essentially a kernel compiled straight from an ACK branch. The GKI isolates SoC vendor and OEM customizations to plugin modules, eliminating out-of-tree code and allowing Google to push kernel updates directly to the end-user. For over a year, Google has been working on a way to deliver GKI updates via the Play Store, through the use of a Mainline module.
As a result, devices that launch with Android 12 that run Linux kernel 5.10.43 or higher must do one of the following, according to Mishaal Rahman.
- Deploy a Google-signed boot image
OR
- Deploy a boot image with a kernel that exports a KMI (Kernel Module Interface) that’s a subset of the KMI exported by the GKI, exports a userspace API that is a superset of the UAPI exposed by the GKI, and support all features of the corresponding GKI version
Vendors can create modules that plug into the GKI, but the idea of the GKI is that Google takes on the burden of responsibility for handling kernel changes. The Kernel Module Interface (or KMI, more on this in the later parts of the article) is effectively where out-of-tree code is expected to go.
The Google Pixel 6 series launched with Android 12 out of the box and ships with Linux kernel 5.10, and it’s the first phone to ship with a GKI. Because Google could potentially update the kernel through the Play Store, we might even see frequent kernel updates, as LTS kernel updates are typically released weekly. Either way, it’s a much better system than the currently-cumbersome method of updating via OTA, though this does mean it is inherently tied to the GMS framework.
Google simply defines the GKI as the following:
- It’s built from the ACK sources.
- It’s a single-kernel binary plus associated loadable modules per architecture, per LTS release (currently only arm64 for
android11-5.4andandroid12-5.4). - It’s tested with all Android Platform releases that are supported for the associated ACK. There’s no feature deprecation for the lifetime of a GKI kernel version
- It exposes a stable KMI to drivers within a given LTS.
- It does not contain SoC or board-specific code.
Google even wants to be in a position by 2023 wherein it can take an “upstream first” development model. This will help Google ensure new code lands first in the mainline Linux kernel, reducing “technical debt” accrued out-of-tree code on Android devices.
The Kernel Module Interface (KMI)
The Kernel Module Interface, or KMI, is part of Google’s solution to the ongoing fragmentation in Android. In essence, SoC and board support are no longer located in the core kernel and are instead moved into loadable modules. Both the kernel and modules can be updated independently then, as modules are updated in /lib/modules. The GKI itself is supposed to be as clean and generic as possible, which is made possible by offloading what is now out-of-tree code into separate modules.
As Ted Kjos explained at this year’s Linux Plumbers Conference, “the big multi-year push is to get all of the hardware-specific code out of the generic kernel and into vendor modules. We have to have a stable interface between those vendor modules and the generic kernel so that they can ship asynchronously.” GKI 1.0 is essentially a “compliance test”.
In fact, GKI compatibility means that the device passes the VTS and CTS-on-GSI+GKI tests with the Generic System Image (GSI) and the GKI kernel installed by flashing the GKI boot image into the boot partition and GSI system image in the system partition. The Vendor Test Suite, or VTS, is an automated test that all devices must pass to be considered compatible with Project Treble. The Compatibility Test Suite, or CTS, is required in order to access Google’s suite of applications.
Devices can ship with a different product kernel and can use loadable modules that GKI doesn’t provide. However, both the product and GKI kernels must load modules from the same vendor_boot and vendor partitions. Therefore, all product kernels are required to have the same binary kernel module interface (KMI).
The above diagram shows what Google wants to do, and explains how it intends on reaching that. The Generic Kernel and GKI modules will be a part of AOSP, and the GKI can communicate with the Android framework and the Hardware Abstraction Layer (HAL) that a vendor may implement. The specific proprietary code that a vendor wants in the kernel (for example, camera drivers) will instead be pushed into a vendor module that becomes an extension of the GKI via the KMI.
How the GKI can help solve Android’s fragmentation problem
Google has been putting a lot of work into streamlining the development process of smartphones. Every OEM wants its own brand identity, and every OEM wants to be able to have ownership of its devices. Unlike the Android One program, Android smartphones can pretty much be whatever they want, so long as they adhere to the set of rules that Google sets out in order to receive a GMS license. However, in the past, Google hasn’t done a whole lot to reign in Android device development, with changes such as Project Treble, Mainline, and now the GKI being a lot more recent in Android’s history.
But will it help? It should do, though it’s likely to be a multi-year affair that bears visible fruit later down the line. This will only apply to devices that launch with Android 12, meaning that we’re going to see devices that don’t have a GKI for years to come. That was also a criticism of Project Treble when that was announced, though obviously all devices launched nowadays support it. These things take time, and as Google slowly draws the reigns on Android, the development process is eased for all of the OEMs in the Android ecosystem, even if some of them would rather retain full control over the Linux kernel that’s used on Android smartphones.
The post Google’s Generic Kernel Image is the next step towards solving Android’s fragmentation problem appeared first on xda-developers.
from xda-developers https://ift.tt/3FEUcnH
via IFTTT
