Gradle recently rolled out a new feature called the “configuration cache” aimed at “significantly improving build performance by caching the result of the configuration phase and reusing it for subsequent builds. Using the configuration cache, Gradle can skip the configuration phase entirely when nothing that affects the build configuration, such as build scripts, has changed.”
One requirement of enabling the configuration cache is that your project must not register any BuildListeners as these cannot be serialised to disk and thus, cached. As a result, it leaves developers stuck between a rock and a hard place, as they have to decide if they want to sacrifice build speed for build time information or vice versa.
In this blog post I will be talking about how you can enable the configuration cache and capture build data utilising Gradle BuildServices and their internal APIs, for which there are little resources online in achieving.
Introducing Build Services
Build services provide an alternative mechanism for hooking into a Gradle build and receiving information about task execution and operation completion. And most importantly, they’re configuration cacheable.
Capturing Build Time Information
To capture build time information we create a build service that implements Gradle’s internal BuildOperationListener interface. When BuildOperationListener#finished is called with the RunRootBuildWorkBuildOperationType the build has finished and the build start time is provided via its Details.
| abstract class BuildDurationService : BuildService<BuildServiceParameters.None>, BuildOperationListener, AutoCloseable { | |
| var buildDuration: Long? = null | |
| override fun started(p0: BuildOperationDescriptor, p1: OperationStartEvent) {} | |
| override fun progress(p0: OperationIdentifier, p1: OperationProgressEvent) {} | |
| override fun finished(buildOperationDescriptor: BuildOperationDescriptor, operationFinishEvent: OperationFinishEvent) { | |
| if (buildOperationDescriptor.details is RunRootBuildWorkBuildOperationType.Details) { | |
| val details = buildOperationDescriptor.details as RunRootBuildWorkBuildOperationType.Details? | |
| details?.buildStartTime?.let { buildStartTime -> | |
| buildDuration = System.currentTimeMillis() - buildStartTime | |
| } | |
| } | |
| } | |
| } |
Next, to capture configuration time and confirmation the configuration phase finished successfully, we use the start time of the first task provided by operationFinishEvent and subtract from it the start time of the build.
| abstract class BuildDurationService : BuildService<BuildServiceParameters.None>, BuildOperationListener, AutoCloseable { | |
| var buildDuration: Long? = null | |
| var configurationDuration: Long? = null | |
| var configurationPhaseFailed = true | |
| override fun started(p0: BuildOperationDescriptor, p1: OperationStartEvent) {} | |
| override fun progress(p0: OperationIdentifier, p1: OperationProgressEvent) {} | |
| override fun finished(buildOperationDescriptor: BuildOperationDescriptor, operationFinishEvent: OperationFinishEvent) { | |
| if (buildOperationDescriptor.details is RunRootBuildWorkBuildOperationType.Details) { | |
| /** | |
| * Runs when build phase finishes, therefore we can assume configuration phase passed | |
| */ | |
| configurationPhaseFailed = false | |
| val details = buildOperationDescriptor.details as RunRootBuildWorkBuildOperationType.Details? | |
| details?.buildStartTime?.let { buildStartTime -> | |
| buildDuration = System.currentTimeMillis() - buildStartTime | |
| val firstTaskStartTime = operationFinishEvent.startTime | |
| this.configurationDuration = firstTaskStartTime - buildStartTime | |
| } | |
| } | |
| } | |
| } |
Once the BuildDurationService is created, to start receiving BuildOperationListener callbacks, we register the service with Gradle’s BuildEventListenerRegistryInternal.
| class BuildTimeLoggerPlugin : Plugin<Project> { | |
| override fun apply(project: Project) { | |
| val gradle = project.gradle | |
| val buildDurationService = registerBuildDurationService(gradle) | |
| } | |
| private fun registerBuildDurationService(gradle: Gradle): Provider<BuildDurationService> { | |
| val registry = gradle.serviceRegistry()[BuildEventListenerRegistryInternal::class.java] | |
| val buildDurationService = gradle.sharedServices.registerIfAbsent("build-duration-service", BuildDurationService::class.java) { } | |
| registry.onOperationCompletion(buildDurationService) | |
| return buildDurationService | |
| } | |
| } | |
| fun Gradle.serviceRegistry(): ServiceRegistry = (this as DefaultGradle).services |
Capturing Task Execution Information
Similarly to capturing build time duration, Gradle allows you to register a service to listen to task completion events via OperationCompletionListener callbacks.
Filtering each event’s display name will provide their task execution result and (if any) a build failure message.
| abstract class BuildTaskService : BuildService<BuildServiceParameters.None>, OperationCompletionListener { | |
| var fromCacheTasksCount = 0 | |
| var upToDateTasksCount = 0 | |
| var executedTasksCount = 0 | |
| var buildPhaseFailureMessage: String? = null | |
| val buildPhaseFailed: Boolean | |
| get() = buildPhaseFailureMessage != null | |
| override fun onFinish(event: FinishEvent?) { | |
| if (event == null || event !is TaskFinishEvent) | |
| return | |
| when { | |
| event.isFromCache() -> { | |
| fromCacheTasksCount++ | |
| } | |
| event.isUpToDate() -> { | |
| upToDateTasksCount++ | |
| } | |
| event.isSuccess() -> { | |
| executedTasksCount++ | |
| } | |
| } | |
| if (event.result is TaskFailureResult) { | |
| buildPhaseFailureMessage = | |
| (event.result as TaskFailureResult).failures.firstOrNull()?.message | |
| ?: "${event.displayName} Failed without message" | |
| } | |
| } | |
| private fun FinishEvent.isUpToDate(): Boolean { | |
| return this.displayName.endsWith("UP-TO-DATE") | |
| } | |
| private fun FinishEvent.isFromCache(): Boolean { | |
| return this.displayName.endsWith("FROM-CACHE") | |
| } | |
| private fun FinishEvent.isSuccess(): Boolean { | |
| return this.displayName.endsWith("SUCCESS") | |
| } | |
| } |
Job Offers
OUR VIDEO RECOMMENDATION
Jobs
To start receiving callbacks we register the service as a task completion listener.
| class BuildTimeLoggerPlugin : Plugin<Project> { | |
| override fun apply(project: Project) { | |
| val gradle = project.gradle | |
| val buildTaskService = registerBuildTaskService(gradle) | |
| } | |
| private fun registerBuildTaskService(gradle: Gradle): Provider<BuildTaskService> { | |
| val registry = gradle.serviceRegistry()[BuildEventListenerRegistryInternal::class.java] | |
| val buildTaskService = gradle.sharedServices.registerIfAbsent("build-task-service", BuildTaskService::class.java) { } | |
| registry.onTaskCompletion(buildTaskService) | |
| return buildTaskService | |
| } | |
| } | |
| fun Gradle.serviceRegistry(): ServiceRegistry = (this as DefaultGradle).services |
Reporting Data
To report this data we will create another build service to isolate the functionality between the generation and reporting, parsing the services as arguments to access their data. Implementing the AutoCloseable interface provides a close function to trigger the report, with it being called at the end of the build. And finally, we implement the BuildOperationListener with empty functions to ensure the build service is initialised during the build and that the AutoCloseable#onClose is executed (AutoCloseable#onClose is executed when the service is disposed of, because this service isn’t actually being used by any tasks we need to give Gradle another reason to initialise the service, hence registering it for build operation callbacks).
| abstract class BuildReporterService : BuildService<BuildReporterService.Params>, BuildOperationListener, AutoCloseable { | |
| interface Params : BuildServiceParameters { | |
| fun getBuildDurationServiceProvider(): Property<Provider<BuildDurationService>> | |
| fun getBuildTaskServiceProvider(): Property<Provider<BuildTaskService>> | |
| } | |
| override fun started(p0: BuildOperationDescriptor, p1: OperationStartEvent) {} | |
| override fun progress(p0: OperationIdentifier, p1: OperationProgressEvent) {} | |
| override fun finished(buildOperationDescriptor: BuildOperationDescriptor, operationFinishEvent: OperationFinishEvent) {} | |
| override fun close() { | |
| logBuildStats() | |
| } | |
| private fun logBuildStats() { | |
| val buildDurationService = parameters.getBuildDurationServiceProvider().get().get() | |
| val buildTaskService = parameters.getBuildTaskServiceProvider().get().get() | |
| val buildReport = getBuildReport(buildDurationService, buildTaskService) | |
| val logger = Logging.getLogger("console-logger") | |
| logger.lifecycle(buildReport.toString()) | |
| } | |
| private fun getBuildReport( | |
| buildDurationService: BuildDurationService, | |
| buildTaskService: BuildTaskService, | |
| ): BuildReport { | |
| val taskExecutionStatistics = getTasksExecutionStatistics(buildTaskService) | |
| val buildFailureMessage = if (buildTaskService.buildPhaseFailed) { | |
| buildTaskService.buildPhaseFailureMessage ?: "Build failed" | |
| } else if (buildDurationService.configurationPhaseFailed) { | |
| "Configuration Phase Failed" | |
| } else { | |
| null | |
| } | |
| val buildFailed = (buildFailureMessage != null) | |
| return BuildReport( | |
| totalElapsedBuildTimeMs = buildDurationService.buildDuration, | |
| configurationTimeMs = buildDurationService.configurationDuration, | |
| taskExecutionStatistics = taskExecutionStatistics, | |
| buildFailed = buildFailed, | |
| buildFailureMessage = buildFailureMessage | |
| ) | |
| } | |
| private fun getTasksExecutionStatistics(buildTaskService: BuildTaskService): TaskExecutionStatistics { | |
| return TaskExecutionStatistics( | |
| buildTaskService.executedTasksCount, | |
| buildTaskService.fromCacheTasksCount, | |
| buildTaskService.upToDateTasksCount | |
| ) | |
| } | |
| } |
Finally, we register the build service and parse the other services as parameter.
| class BuildTimeLoggerPlugin : Plugin<Project> { | |
| override fun apply(project: Project) { | |
| val gradle = project.gradle | |
| val buildDurationService = registerBuildDurationService(gradle) | |
| val buildTaskService = registerBuildTaskService(gradle) | |
| registerBuildReporterService(gradle, buildDurationService, buildTaskService) | |
| } | |
| private fun registerBuildReporterService( | |
| gradle: Gradle, | |
| buildDurationService: Provider<BuildDurationService>, | |
| buildTaskService: Provider<BuildTaskService> | |
| ): Provider<BuildReporterService> { | |
| val registry = gradle.serviceRegistry()[BuildEventListenerRegistryInternal::class.java] | |
| val buildReporterService = gradle.sharedServices.registerIfAbsent("build-reporter-service", BuildReporterService::class.java) { service -> | |
| service.parameters.getBuildDurationServiceProvider().set(buildDurationService) | |
| service.parameters.getBuildTaskServiceProvider().set(buildTaskService) | |
| } | |
| registry.onOperationCompletion(buildReporterService) // gives gradle a reason to instantiate the build service and call onClose at the end of the build | |
| return buildReporterService | |
| } | |
| private fun registerBuildDurationService(gradle: Gradle) { ... } | |
| private fun registerBuildTaskService(gradle: Gradle) { ... } | |
| } | |
| fun Gradle.serviceRegistry(): ServiceRegistry = (this as DefaultGradle).services |
If all has been done correctly you should see an output in your Build Output window similar to the following:
BuildReport(totalElapsedBuildTimeMs=23525, configurationTimeMs=8877, taskExecutionStatistics=org.gradle.api.internal.tasks.execution.statistics.TaskExecutionStatistics@6a5aabca, buildFailed=false, buildFailureMessage=null, buildTaskNames=:app:assembleDebug, gradleVersion=7.4.2, daemonsRunning=1)
Parsing Cacheable Values
You may also want to parse additional values to a build service such as the task names of the build, Gradle version etc. However, before you do it’s important to know these values will be provided by the configuration cache on builds when the cache is used so don’t parse any values that could change in between cached builds such as the number of Gradle daemons running.
To parse cacheable values you add additional get functions to the BuildServiceParameters interface defined in your service. Note: The function names must start with “get” and return the type wrapped in a Property.
| abstract class BuildReporterService : BuildService<BuildReporterService.Params>, AutoCloseable { | |
| interface Params : BuildServiceParameters { | |
| fun getBuildTaskNames(): Property<String> | |
| fun getGradleVersion(): Property<String> | |
| } | |
| override fun close() { | |
| val buildTaskNames = parameters.getBuildTaskNames().get() | |
| val gradleVersion = parameters.getGradleVersion().get() | |
| } | |
| } |
Cached values are then set when registering the service.
| val buildReporterService = gradle.sharedServices.registerIfAbsent("build-reporter-service", BuildReporterService::class.java) { service -> | |
| service.parameters.getBuildTaskNames().set(gradle.startParameter.taskNames.joinToString()) | |
| service.parameters.getGradleVersion().set(gradle.gradleVersion) | |
| } |
Caveats
- There are numerous APIs used throughout that are either marked as unstable or live in an internal Gradle directory. With this comes increased risks of these APIs changing or being removed entirely in the future.
- Configuration time is not reported when the configuration phase fails. This is due to the fact that the start build time is acquired from the root task, if the configuration phase fails the root task won’t execute and no start time for us.
You can find a working project with all of the above code in my Github.
Happy building…
Shout out to Michael Tweed and Alejandro Rosas for proof reading and feedback.
This article was originally published on proandroiddev.com on June 21, 2022
