Blog Infos
Author
Published
Topics
Published
Topics

Gradle recently rolled out a new feature called the “configuration cache” aimed at “significantly improving build performance by caching the result of the configuration phase and reusing it for subsequent builds. Using the configuration cache, Gradle can skip the configuration phase entirely when nothing that affects the build configuration, such as build scripts, has changed.”

One requirement of enabling the configuration cache is that your project must not register any BuildListeners as these cannot be serialised to disk and thus, cached. As a result, it leaves developers stuck between a rock and a hard place, as they have to decide if they want to sacrifice build speed for build time information or vice versa.

In this blog post I will be talking about how you can enable the configuration cache and capture build data utilising Gradle BuildServices and their internal APIs, for which there are little resources online in achieving.

Introducing Build Services

Build services provide an alternative mechanism for hooking into a Gradle build and receiving information about task execution and operation completion. And most importantly, they’re configuration cacheable.

Capturing Build Time Information

To capture build time information we create a build service that implements Gradle’s internal BuildOperationListener interface. When BuildOperationListener#finished is called with the RunRootBuildWorkBuildOperationType the build has finished and the build start time is provided via its Details.

abstract class BuildDurationService : BuildService<BuildServiceParameters.None>, BuildOperationListener, AutoCloseable {
var buildDuration: Long? = null
override fun started(p0: BuildOperationDescriptor, p1: OperationStartEvent) {}
override fun progress(p0: OperationIdentifier, p1: OperationProgressEvent) {}
override fun finished(buildOperationDescriptor: BuildOperationDescriptor, operationFinishEvent: OperationFinishEvent) {
if (buildOperationDescriptor.details is RunRootBuildWorkBuildOperationType.Details) {
val details = buildOperationDescriptor.details as RunRootBuildWorkBuildOperationType.Details?
details?.buildStartTime?.let { buildStartTime ->
buildDuration = System.currentTimeMillis() - buildStartTime
}
}
}
}

Next, to capture configuration time and confirmation the configuration phase finished successfully, we use the start time of the first task provided by operationFinishEvent and subtract from it the start time of the build.

abstract class BuildDurationService : BuildService<BuildServiceParameters.None>, BuildOperationListener, AutoCloseable {
var buildDuration: Long? = null
var configurationDuration: Long? = null
var configurationPhaseFailed = true
override fun started(p0: BuildOperationDescriptor, p1: OperationStartEvent) {}
override fun progress(p0: OperationIdentifier, p1: OperationProgressEvent) {}
override fun finished(buildOperationDescriptor: BuildOperationDescriptor, operationFinishEvent: OperationFinishEvent) {
if (buildOperationDescriptor.details is RunRootBuildWorkBuildOperationType.Details) {
/**
* Runs when build phase finishes, therefore we can assume configuration phase passed
*/
configurationPhaseFailed = false
val details = buildOperationDescriptor.details as RunRootBuildWorkBuildOperationType.Details?
details?.buildStartTime?.let { buildStartTime ->
buildDuration = System.currentTimeMillis() - buildStartTime
val firstTaskStartTime = operationFinishEvent.startTime
this.configurationDuration = firstTaskStartTime - buildStartTime
}
}
}
}

Once the BuildDurationService is created, to start receiving BuildOperationListener callbacks, we register the service with Gradle’s BuildEventListenerRegistryInternal.

class BuildTimeLoggerPlugin : Plugin<Project> {
override fun apply(project: Project) {
val gradle = project.gradle
val buildDurationService = registerBuildDurationService(gradle)
}
private fun registerBuildDurationService(gradle: Gradle): Provider<BuildDurationService> {
val registry = gradle.serviceRegistry()[BuildEventListenerRegistryInternal::class.java]
val buildDurationService = gradle.sharedServices.registerIfAbsent("build-duration-service", BuildDurationService::class.java) { }
registry.onOperationCompletion(buildDurationService)
return buildDurationService
}
}
fun Gradle.serviceRegistry(): ServiceRegistry = (this as DefaultGradle).services
Capturing Task Execution Information

Similarly to capturing build time duration, Gradle allows you to register a service to listen to task completion events via OperationCompletionListener callbacks.

Filtering each event’s display name will provide their task execution result and (if any) a build failure message.

abstract class BuildTaskService : BuildService<BuildServiceParameters.None>, OperationCompletionListener {
var fromCacheTasksCount = 0
var upToDateTasksCount = 0
var executedTasksCount = 0
var buildPhaseFailureMessage: String? = null
val buildPhaseFailed: Boolean
get() = buildPhaseFailureMessage != null
override fun onFinish(event: FinishEvent?) {
if (event == null || event !is TaskFinishEvent)
return
when {
event.isFromCache() -> {
fromCacheTasksCount++
}
event.isUpToDate() -> {
upToDateTasksCount++
}
event.isSuccess() -> {
executedTasksCount++
}
}
if (event.result is TaskFailureResult) {
buildPhaseFailureMessage =
(event.result as TaskFailureResult).failures.firstOrNull()?.message
?: "${event.displayName} Failed without message"
}
}
private fun FinishEvent.isUpToDate(): Boolean {
return this.displayName.endsWith("UP-TO-DATE")
}
private fun FinishEvent.isFromCache(): Boolean {
return this.displayName.endsWith("FROM-CACHE")
}
private fun FinishEvent.isSuccess(): Boolean {
return this.displayName.endsWith("SUCCESS")
}
}

Job Offers

Job Offers


    Senior Android Developer

    Komoot
    remote
    • Full Time
    apply now

    Android Engineer

    American Express
    New York
    • Full Time
    apply now

    Softwareentwickler Java und Kotlin / Android (w/m/d)

    AVM GmbH
    Berlin
    • Full Time
    apply now
Load more listings

OUR VIDEO RECOMMENDATION

,

Writing backwards-compatible Gradle plugins

Gradle plugins are powerful – you can integrate custom logic into the build process to make it fit your needs. In the Android world, most Gradle plugins will have to interact with the Android Gradle…
Watch Video

Writing backwards-compatible Gradle plugins

SIMON SCHILLER
Software Engineer
Spotify

Writing backwards-compatible Gradle plugins

SIMON SCHILLER
Software Engineer
Spotify

Writing backwards-compatible Gradle plugins

SIMON SCHILLER
Software Engineer
Spotify

Jobs

To start receiving callbacks we register the service as a task completion listener.

class BuildTimeLoggerPlugin : Plugin<Project> {
override fun apply(project: Project) {
val gradle = project.gradle
val buildTaskService = registerBuildTaskService(gradle)
}
private fun registerBuildTaskService(gradle: Gradle): Provider<BuildTaskService> {
val registry = gradle.serviceRegistry()[BuildEventListenerRegistryInternal::class.java]
val buildTaskService = gradle.sharedServices.registerIfAbsent("build-task-service", BuildTaskService::class.java) { }
registry.onTaskCompletion(buildTaskService)
return buildTaskService
}
}
fun Gradle.serviceRegistry(): ServiceRegistry = (this as DefaultGradle).services
Reporting Data

To report this data we will create another build service to isolate the functionality between the generation and reporting, parsing the services as arguments to access their data. Implementing the AutoCloseable interface provides a close function to trigger the report, with it being called at the end of the build. And finally, we implement the BuildOperationListener with empty functions to ensure the build service is initialised during the build and that the AutoCloseable#onClose is executed (AutoCloseable#onClose is executed when the service is disposed of, because this service isn’t actually being used by any tasks we need to give Gradle another reason to initialise the service, hence registering it for build operation callbacks).

abstract class BuildReporterService : BuildService<BuildReporterService.Params>, BuildOperationListener, AutoCloseable {
interface Params : BuildServiceParameters {
fun getBuildDurationServiceProvider(): Property<Provider<BuildDurationService>>
fun getBuildTaskServiceProvider(): Property<Provider<BuildTaskService>>
}
override fun started(p0: BuildOperationDescriptor, p1: OperationStartEvent) {}
override fun progress(p0: OperationIdentifier, p1: OperationProgressEvent) {}
override fun finished(buildOperationDescriptor: BuildOperationDescriptor, operationFinishEvent: OperationFinishEvent) {}
override fun close() {
logBuildStats()
}
private fun logBuildStats() {
val buildDurationService = parameters.getBuildDurationServiceProvider().get().get()
val buildTaskService = parameters.getBuildTaskServiceProvider().get().get()
val buildReport = getBuildReport(buildDurationService, buildTaskService)
val logger = Logging.getLogger("console-logger")
logger.lifecycle(buildReport.toString())
}
private fun getBuildReport(
buildDurationService: BuildDurationService,
buildTaskService: BuildTaskService,
): BuildReport {
val taskExecutionStatistics = getTasksExecutionStatistics(buildTaskService)
val buildFailureMessage = if (buildTaskService.buildPhaseFailed) {
buildTaskService.buildPhaseFailureMessage ?: "Build failed"
} else if (buildDurationService.configurationPhaseFailed) {
"Configuration Phase Failed"
} else {
null
}
val buildFailed = (buildFailureMessage != null)
return BuildReport(
totalElapsedBuildTimeMs = buildDurationService.buildDuration,
configurationTimeMs = buildDurationService.configurationDuration,
taskExecutionStatistics = taskExecutionStatistics,
buildFailed = buildFailed,
buildFailureMessage = buildFailureMessage
)
}
private fun getTasksExecutionStatistics(buildTaskService: BuildTaskService): TaskExecutionStatistics {
return TaskExecutionStatistics(
buildTaskService.executedTasksCount,
buildTaskService.fromCacheTasksCount,
buildTaskService.upToDateTasksCount
)
}
}

Finally, we register the build service and parse the other services as parameter.

class BuildTimeLoggerPlugin : Plugin<Project> {
override fun apply(project: Project) {
val gradle = project.gradle
val buildDurationService = registerBuildDurationService(gradle)
val buildTaskService = registerBuildTaskService(gradle)
registerBuildReporterService(gradle, buildDurationService, buildTaskService)
}
private fun registerBuildReporterService(
gradle: Gradle,
buildDurationService: Provider<BuildDurationService>,
buildTaskService: Provider<BuildTaskService>
): Provider<BuildReporterService> {
val registry = gradle.serviceRegistry()[BuildEventListenerRegistryInternal::class.java]
val buildReporterService = gradle.sharedServices.registerIfAbsent("build-reporter-service", BuildReporterService::class.java) { service ->
service.parameters.getBuildDurationServiceProvider().set(buildDurationService)
service.parameters.getBuildTaskServiceProvider().set(buildTaskService)
}
registry.onOperationCompletion(buildReporterService) // gives gradle a reason to instantiate the build service and call onClose at the end of the build
return buildReporterService
}
private fun registerBuildDurationService(gradle: Gradle) { ... }
private fun registerBuildTaskService(gradle: Gradle) { ... }
}
fun Gradle.serviceRegistry(): ServiceRegistry = (this as DefaultGradle).services

If all has been done correctly you should see an output in your Build Output window similar to the following:

BuildReport(totalElapsedBuildTimeMs=23525, configurationTimeMs=8877, taskExecutionStatistics=org.gradle.api.internal.tasks.execution.statistics.TaskExecutionStatistics@6a5aabca, buildFailed=false, buildFailureMessage=null, buildTaskNames=:app:assembleDebug, gradleVersion=7.4.2, daemonsRunning=1)
Parsing Cacheable Values

You may also want to parse additional values to a build service such as the task names of the build, Gradle version etc. However, before you do it’s important to know these values will be provided by the configuration cache on builds when the cache is used so don’t parse any values that could change in between cached builds such as the number of Gradle daemons running.

To parse cacheable values you add additional get functions to the BuildServiceParameters interface defined in your service. Note: The function names must start with “get” and return the type wrapped in a Property.

abstract class BuildReporterService : BuildService<BuildReporterService.Params>, AutoCloseable {
interface Params : BuildServiceParameters {
fun getBuildTaskNames(): Property<String>
fun getGradleVersion(): Property<String>
}
override fun close() {
val buildTaskNames = parameters.getBuildTaskNames().get()
val gradleVersion = parameters.getGradleVersion().get()
}
}

Cached values are then set when registering the service.

val buildReporterService = gradle.sharedServices.registerIfAbsent("build-reporter-service", BuildReporterService::class.java) { service ->
service.parameters.getBuildTaskNames().set(gradle.startParameter.taskNames.joinToString())
service.parameters.getGradleVersion().set(gradle.gradleVersion)
}
Caveats
  • There are numerous APIs used throughout that are either marked as unstable or live in an internal Gradle directory. With this comes increased risks of these APIs changing or being removed entirely in the future.
  • Configuration time is not reported when the configuration phase fails. This is due to the fact that the start build time is acquired from the root task, if the configuration phase fails the root task won’t execute and no start time for us.

You can find a working project with all of the above code in my Github.

Happy building…

Shout out to Michael Tweed and Alejandro Rosas for proof reading and feedback.

 

This article was originally published on proandroiddev.com on June 21, 2022

YOU MAY BE INTERESTED IN

YOU MAY BE INTERESTED IN

blog
Managing dependencies in a single module project is pretty simple, but when you start…
READ MORE
blog

Running Instrumented Tests in a Gradle task

During the latest Google I/O, a lot of great new technologies were shown. The…
READ MORE
blog
Mobile device concept is one step ahead of the trend in technology, bringing an…
READ MORE
blog
According to the Gradle documentation dependency resolution is a process that consists of two…
READ MORE

Leave a Reply

Your email address will not be published.

Fill out this field
Fill out this field
Please enter a valid email address.

Menu