hey everyone welcome to the presentation perspectives from a solutions engineer by me uh nelson osaki so um yeah i'm just gonna give you a perspective on what i do and what it is like to be a gradle solutions engineer and of course since this is a presentation from gradle i'm getting some really weird feedback

i'm hearing an echo of the presentation it's really distracting

how do i turn this off

okay wait if i keep talking the promise i hear yeah i hear myself okay um

i maybe i'll just mute my audio okay i'm just going to mute my audio and pretend keep everything going

uh okay cool so yeah perspectives from a solutions engineer this talk is going to cover a couple of things why is productivity important uh what do i do as a solutions engineer at gradle of course this is a gradle presentation you all are here because you want to know how to make your build faster and hopefully my perspective on as a solutions engineer will give you some insights on that and i'll also answer what is coming up in gradle to make your build faster so let's get started a little background on me so i was previously an android engineer i was in your shoes i've worked on large projects such as soundcloud i've worked at square i've also worked at small startups that you've probably never heard of and right now i'm a solutions engineer at gradle so i used to be a customer of gradle enterprise and gradle and i used to use it all myself i still do uh but now i help everybody with their android builds i'm also a gradle plugin maintainers so i've built some gradle plugins uh i won't go into those during the talk but if you're curious i developed flagle which allows you to easily scale your instrumentation tests on firebase test lab delect which automatically replaces dagger with dagger reflect and i also work on a gradle plugin called the gradle doctor which gives you actionable insights for your gradle build um so i work on the gradle enterprise part of gradle um and uh but yeah so what what is gradle enterprise so there's gradle build tool which is the open source build tool that everyone is familiar with and there's also gradle enterprise which helps you scale up your gradle build and i'll go into some details on exactly what gradle enterprise is how it differs from the gradle build tool and how it's helpful in large builds but just to clarify some things because it often gets confusing there's a lot of different companies at play uh with gradle with android and all those things so so google is the team that builds the android gradle plugin that's completely maintained within google of course we collaborate on all these things but google also builds android studio which is built on intellij which is developed by jetbrains jetbrains builds intellij they also build the kotlin language so there's of course a very close collaboration between all three of us but those things are controlled within those respective companies so gradle works on gradle build tool itself and gradle enterprise google works on the android gradle plugin and android studio which is the specific features related to android within uh android studio whereas intellij and kotlin are developed by jetbrains uh so next why is productivity important um productivity is really important because when you're an android developer you spend a lot of time building and um yeah so when you're developing features uh on an android team you know you you have to see get your feedback and so you get your feedback you you you want to and yeah to to get your feedback as soon as possible of course there's all sorts of tech things that get in the way of your your productivity of course if your code base is a mess that slows you down if your builds are slow that slows you down if um you know there's any any sorts of things that you should be working on that aren't part of the task that you that you're doing that's that's tech debt of course there are other parts of building you know android apps you have to work with designers build the product features work with users things like that but i'm just kind of simplifying things for the purposes of this presentation so if you're working on tech debt of course you can refactor code to make it simpler but you can also improve the the build speed and so i'm here to tell you that build speed is tech debt uh but it always pays off because uh if you make your build faster you always get to use that whereas if you're in a part of the code base that you you refactor it and you make it beautiful but then you never touch that code again well what was the point of that refactoring so to give you a perspective on on how much time is is spent building let's say you had 60 seconds of waste in in your build and you spend 50 you you build 50 times per day times 50 developers that the equals per week 42 hours lost so you know your team might be smaller your team might be bigger you might not have so much waste you might not build so many times per day but that just gives you a calculation of how many hours you could be losing per week and we have a more in-depth version of this calculation online on gradle.com roi calculator if you're curious this will help you calculate not just in terms of hours loss but also uh money lost for your company or your team uh with with your builds uh so fast builds really really matter and of course there's the money part but it's also motivation right the slower your build is the the less motivated you are to fix stuff um and so what what this means basically is that if if you had this this loss per uh if you're losing 42 hours per per day that means that you could basically hire five new people on your team without paying you know any of the recruiting costs any of their salaries anything like that so if you just recuperate that in terms of build speeds that make your team much more productive uh so this is why it's very easy to justify working on build speed as tech that as opposed to refactoring parts of your code base so that motivates you let's let's get started on uh yeah what do i do um so okay i work with trial prospects in gradle enterprise to help them improve their build so if you're if you're motivated by the fact that your build could be potentially faster or your build is really slow and you know it can be slower i work with customers who fill out trial form and yeah basically make their builds faster it's pretty fun and to get the most out of gradle enterprise uh i also help gradle address customers needs in general so it's not just about me helping customers but also filtering feedback to gradle you know in slack with company priorities things like that because as a solutions engineer i have the closest connection to the customers as well as the android community and i see a lot of builds many different kinds of builds i see android builds i see gradle builds i also see maven builds gradle enterprise works with maven so uh that's also a thing i help optimize for our customers and we see all kinds of bills the teams from 10 person small android shops startups all the way to thousand person teams and we help all of them

so yeah i help customers optimize and understand their builds also reduce failures using our failure dashboards which i'll show later on in the presentation i preach developer productivity to the teams to motivate them how important this is and yeah help customers maximize their trial experience with gradle enterprise um another thing we do is we present business cases for purchasing gradle enterprise uh because yeah so gradle enterprise is a paid product we we help make the business case because you're probably an engineer you're probably not so uh yeah the business and dealing with money whatever we so we we help you out with that as customer as uh solutions engineers i also work closely with sales and we help all of our trial customers do that uh so yeah the solutions engineer you play a lot of hats you do a lot of different things uh and it's kind of fun at least i think so well i also meet well not just me but gradle the build tool team solutions team we meet with google we meet with chet brains and sometimes i'm like hey this past week i saw this thing happen to these customers five times let's do something about it um and yeah the more builds i see the more feedback i get the the better i can filter that to google and jetbrains of course i want to say all of these teams gradle jetbrains google we all have a lot of different priorities and it's all just a matter of prioritizing the right issues at the top of the stack um another thing i do as a solutions engineer as well as the rest of my team is we fix issues and open source plugins that affect our customers so caching issues speed issues test configuration avoidance things like that uh because we don't just want to make the issue the builds better specifically for our customers but we want to make the whole ecosystem and make good examples of gradle plug-ins out in the wild um and so our solutions team here at cradle there's four of us we're a pretty small team and we help all of the the customers so there's gary there's daz uh etienne's our boss and he also does trials every once in a while which i think is pretty cool because it keeps the the connection to the customers strong and then gary and das also work on the build tool uh and gradle enterprise to help improve specific things for our customers and i'll get into that uh in in the next slide in our next few slides uh we're also supported by the operations team the build tools team so if you have any questions if you're a customer we yeah we're like that direct connection we pull in other people from the teams to help maintain your gradle enterprise instance build tool support things like that and yeah basically they help everybody else helps me sound like i know what i'm talking about um okay so yeah we do a lot of gradle enterprise trials uh we work with so yeah during a great enterprise trial we work with the champion from you know any any company that wants to that is thinking about purchasing gradle enterprise we call that that guy our guy or gal he or she is our champion he thinks that cruel enterprise is awesome and we help them optimize their build we meet once a week usually and we go over any issues they saw with gradle enterprise with gradle uh we give them performance improvements tips we do experiments on their build to optimize it we examine build scans together and we also just answer any sort of gradle questions they have so it's a fun experience you learn about cradle you learn about optimizing builds i highly recommend it obviously i'm tooting my own horn here but uh it's it's fun and you work with all kinds of people so yeah now now comes the part you've all probably come to this talk for is what do i see having seen a lot of builds what what are common patterns what are common issues right so of course the understanding of gradle can vary wildly some people are experts in gradle and want to dive deep into very specific caching issues other people are much newer to gradle and and any sorts of tips we give them help out a lot and but we're here to help everybody and even gradle experts the the level of optimizations in the bills can vary wildly uh which which is awesome right we want to help everybody and there's always room for improvements no matter how optimized your build already is there's of course diminishing returns so you know you can spend hours optimizing a tiny corner of your build but we want to focus on the most important things first so yeah how do you make your build faster so really it depends on your build every different build has a different issue but um we focus on different parts of of the build there's initially initialization phase of gradle sorry let me slow down here three parts to gradle build there's the initialization phase there's a configuration phase and this execution phase initialization phase is where gradle sets the environment and determines which projects will take part in the gradle build uh there's not much room for optimization here this is just getting everything set up but the second page is configuration this is where all the build scripts of all your projects are executed and the task graph is constructed and configured this is usually a big pain point in android builds because android builds have a lot of tasks that are created a large number of models are created android apps have tons of modules

and lots of different build features that take part in the build so this is one aspect of optimization and then the second is execution so avoiding any tasks that should not be executed is also a big part of optimizing an android build okay so where do you start with optimizing these things basically we go through a build scan and we check for red flags and the question is what is the red flag does it look normal in a build does it not um so one one example of one thing we do in all these trials is we run an experiment if you don't change anything in your build no task should execute so basically if you run assemble debug twice uh all the tasks there no no tasks should run and this is what it looks like in uh in a bill that's correct 58 actionable tasks 58 were up to date but if you're seeing that you know three were executed something like that and your command line then something's up so how do you investigate that well oh sorry and you should do this regularly right you should check to make sure that no tasks are just randomly running so you can use a build scan or the build analyzer tool built into android studio for the purposes of this i'm going to focus on build scans because i'm more familiar with them

but yeah you can also get some of these insights in the android studio build analyzer but to run a build scan what you do is you run a build with scan you can also set up the gradle enterprise plugin to run build scans for all your builds uh and then when you're in a build scan you can filter it the timeline uh for all for example all tests that executed all tests that were success uh by using the filter on the build scans page and once you're there you can see you know which tasks executed so a common problem we see is this crashlytics plugin this is an older version of the crashlytics plugin it's always generating the build id because it doesn't declare any outputs this has been fixed the newer version declares its output so it doesn't happen anymore but if you're running an old version make sure to update it um another thing we see you know there's different types of crash plug-ins there's bug snag it can also cause tasks to run all the time so yeah another way to fix that would be to disable it like so

other things we see a lot is just make sure that all your flags gradle performance flags are enabled there's a parallel flag enable the caching flag enable the daemon and you can see all these switches in the in the grid in the build scans what's enabled what's not so yeah just make sure those are enabled because they make your build faster another thing we see sometimes is like tons of tasks being created in and you can see this in the build scans in the configuration tab uh if there's like 16 000 tasks being created in one build script that's that's a red flag that task is not using configuration avoidance um so what can you do about that you can use uh task configuration avoidance there's the guide and in the documentation i won't go into specifics but you can replace certain api uses with other api use usages and that will lazily create tasks instead of immediately creating them which means that if you're not running or executing that task during the build it won't configure that task and that saves a bunch of time

another red flag is configuration time okay and i say this with kind of a how do you say you know this is kind of a rough very rough guideline but i say configuration time should be roughly five seconds for every 100 project if you're taking 10 seconds per 100 projects that's that's okay if you're taking two seconds for 100 project that's amazing uh but if you're seeing like 20 seconds for a hundred project then that's something to investigate in your build you could speed up your build by uh lowering your configuration time and that could be either using lazy test configuration maybe you have a misbehaving plug-in or something like that uh okay so other other things that can speed up your build uh that we just released which is pretty exciting is uh file system watching this one is like the easiest way to speed up your build you can read the blog post but basically you just add this to your gradle.properties file orgdeckgradle.vfs.watch equals true and your build is magically faster and you can see this in the snapshotting inputs page of your build scans it should reduce the amount of time spent snapshotting task inputs and the way that file system watching works is that instead of discarding all the information on all the file system events that happened at the end of the build if you use the gradle daemon it will keep those events in memory around for the next build so it doesn't use any additional memory because these file system events would be in memory anyways it just doesn't discard them at the end of the build and so if you're using the dame this will speed up your build now if you're using you know ci if you're not using the daemon it's not going to speed up your build but for local builds and this is really an improvement for local incremental builds this should significantly reduce the amount of time spent snapshotting inputs

um other improvements uh the the jettifier is performance is much improved in android gradle plugin 4.1 this is because of some bug fixes around the way that

dependency substitution works so just make sure to upgrade to android gradle plugin 4.1 i i mentioned this too let's all work together to kill jetta fire by upgrading all our libraries so if you're the maintainer of a library let's get rid of it it slows down builds even if you're on android gradle plug-in 4.1 you know if you if you're running a build and you see all these transforms running that's that's the jettifier and it's not just uh slows down your build it also adds a bunch of uh disk usage for caching all your artifacts twice basically every time you upgrade your build source your cradle plug-in anything like that it has to redo all these transformations it takes some time sometimes you don't notice on ci this is adding a ton of time you also don't see this in build scans so um yeah let's get rid of the jettifier if you're stuck on the jetta fire i recommend using this uh gradle plug-in you can run this command can i drop jettifier and it'll tell you you can also jetify libraries if you haven't already done so sorry if you have just one library that isn't gentified you can't upgrade it whatever you can jetify it and then either check it into the file system or upload it to your own nexus repository

all right another thing to speed up your builds is updating all your dependencies uh this this is something we also tell all our trial customers um so all all our teams you know jetbrains google gradle we're all working to make your builds faster all the time there's a lot of hard work being put in there so and i'm just going to give some examples of things we worked on recently in all these latest versions these are the latest versions as i'm giving this talk but you should upgrade to all the latest versions of everything because for example gradle 6.7 includes stable file system watching so that improves the build as i just mentioned by reducing time spent snapshotting inputs there's caching improvements in the android gradle plug-in 4.1 there's better um the jettifier bug fix that doesn't cause all your dependencies to be resolved during the configuration phases there um uh gradle enterprise plug-in 3.5 has compressed scan uploads which speeds up the amount of time spent uploading build scans even though that already happens in the background in kotlin 1.4.10 there's fixes for caching bugs and yeah of course third-party plug-ins more caching incremental annotation processors things like that so just update all the things because we're all our teams here at gradle google jeffree's working hard to make things faster and if you want to know what the latest version of everything is there's a gradle plugin that'll help you out with that so use that uh what else caching so caching is really a big thing we focus on in our trials of course there's the greater local cache there is a remote cache gradle enterprise has a built-in remote cache node and of course you'll get a big benefit of using gradle caching i won't get into the details of caching too much we have some more information in the gradle user guide there's also a gradle training called the build cache deep dive if you click that link and all these links will be available on speaker deck at the end of the talk but if you just go to grill.com training you can sign up for a trading training that will tell you all about the build cash on how you can benefit from it there's also a talk by anaki villar at 3 p.m pacific time tomorrow that'll get into more about solving build cache misses and how they use the gradle remote build cache at uh at tinder so i recommend watching that talk if you want to learn more about caching improvements uh but just a on a basic level if a task has been run before and you're using the the cache and none of the cache inputs change you don't need to rerun that task and that includes compilation generation packaging even tests so if a test has been run before you won't need to rerun that test again you don't need to use any plugins to check if the git commit has changed this is actually just built into gradle and the way it works is by testing the runtime class path so if the runtime class path of the test task did not change then it won't need to rerun that test and of course you can add any inputs to that test if you want to using gradle by let's say you wanted to add the date or something like that as an input to the test task because you use that you can add that as an input and then that'll also be part of the cache key

but since you know i'm just going to give a very very quick overview of cache misses using gradle enterprise this is something we do as part of the trials uh and it's it's tons of fun uh so there are some public instances of gradle enterprise uh in case you're curious you know what is this funny gradle enterprise thing what is it what is the difference between gradle enterprise and build scans so i recommend checking these out so the first one is gradle that's uh i don't know if you know but gradle itself uses gradle enterprise you can check out the uh gradle enterprise uh instance at ge.gradle.org the spring boot team also uses gradle enterprise chat unit team uses greater on price and jetbrains also has their own public gradle enterprise instance that you can check out you can see all the build scans that they're running both locally and on ci and so i'm just going to do a really quick task input comparison um oh wait i need to just share a different window so i ran this build earlier today and i just wanted to show really quick how how gradle enterprise works so let's say you wanted to solve a cache miss using gradle enterprise i have this build scan that i ran earlier today and i want to compare it to the build scan that seeded the cache node so what i do is i click this link what this does is it filters all the cache it filters all the build scans by scans with the same git commit id and what and so that means that all these should have the same inputs and if i look for scans that have the same task and maybe we're run on ci so this one down here this was run on c i can compare my build to the one that ran on ci and if i hit the compare button i can see all the inputs to the task that were different in this case surprisingly enough there are a bunch of cache misses here that i should look into there's some arguments to the compile task the groovy compile is using a different jar that's causing a cache miss so even even at gradle here just by simply turning on the cache we're not getting the most benefit from the remote build cache so yeah this build on the right was what seated the build cache i can see here what the differences all these task inputs are and yeah this is this is one of the cool fancy features of gradle enterprise i just wanted to give a quick little overview on that um yeah switching back to the

presentation where did my presentation

go

yeah okay i'm uh starting to run out of time so i'm just going to try and go this quicker there's also failure analysis in gradle enterprise there's flaky test analysis this one's actually kind of cool so i'm gonna share that really quick as well um so let's say you have a bunch of flaky tests in gradle enterprise you can it'll automatically group the flaky test for you and it'll tell you what the reason for the flaky test is so you can see if if you're having different reasons for flaky test failures you can see the error message of what the failure was and you can also compare to all outcomes so when the build is successful when it's flaky when it passes when it's skipped things like that all right going back to the presentation

so many things all right so the things you're all curious about what are the upcoming things that are coming to gradle to make it faster of course there's a configuration cache this is the biggest i shouldn't say the biggest but this is one that's really really exciting especially in android builds if you have hundreds of modules and we've worked closely with a lot of our customers to really make this work at large large builds so if you have a long configuration phase you know 40 50 seconds and you have hundreds of modules the configuration cache will really significantly reduce that time instead of the way gradle works right now as i was talking about there's optimizations you can perform with the execution phase execution phase the configuration phase things like that with the configuration cache if you have run that same exact configuration phase before it'll just take that configuration out of the build cache instead of re-executing that configuration on every single build it's pretty cool there's more information in our user guide it's not uh it is you know it's still an experimental feature but in kotlin 1.4.20 and android gradle plug-in 4.2.0 once these are stable it should be pretty awesome for android builds of course you can check it out now by following the instructions in the blog but once once these are stable pretty much uh most of the features in the android tool change should be configuration cache compatible um yeah there's also bug fixes those those are boring you can flip through these slides later there's some improvements in gradle 6.8 i'm kind of flipping through this because i want to leave some time for questions but this one comes up a lot during trials people like all right should i convert my groovy scripts to kts for and the answer is kind of two two phase um kotlin scripts are slower to compile there usually it takes about one second to compile a kotlin build script whereas a groovy build script takes 10 milliseconds so that's 100 fold difference but there's really significant performance improvements to kotlin build script compilation in gradle 6.8 so that's really exciting it also uses less memory there's also compile avoidance for kts build script compilation so if you change your build source in a non-abi compatible way that'll also uh you know avoid recompiling the build scripts there's configuration for cache for included builds there's okay yeah more stuff there's a fix for unbounded thread executor used during transformations uh there's also in android gradle plug-in 4.2 there's big improvements to android studio sync i know that's a big pain point with all the customers so look out for that google and gradle have been hard at work on that another big fix not in gradle 6.8 but um if you have empty source directory cache misses so if you're using gradle enterprise and you came across one of these cache misses where you had just an empty source directory um and that would cause you not to be able to use that cash output there is a fix coming up for this by gary hale uh who's on the solutions team and i'm really excited about that because that one really bothered me as an engineer so look out for that in an upcoming version of gradle there's also some ease of use improvements coming to gradle so people always ask how should i structure my project things like that um if you that we have this gendrik is working on an idiomatic gradle uh project so this will tell you you know what are the best practices for structuring my project what's the best way to declare dependencies things like that and this idiomatic gradle on github is where these things are being experimented on iterated on improved on so i recommend checking out that project for the best practices on gradle and some of these things make it into the griddle build tool itself so for example kebab case naming is one central repository declaration this is coming in gradle 6.8 so instead of doing all projects like this you will now be able to um use the dependency resolution management block oh i kind of messed up my slides there whatever and then you'll also be able to execute tests from included builds uh okay i'm gonna flip through these extra resources because i want to leave some time for questions but uh yeah we have trainings on gradle you know look at these slides later once i'm done uh there's also some tools on benchmarking gradle builds you know don't just make changes blindly use the gradle profiler um nate evol has also made a nice blog post on understanding and profiling android gradle builds this is talk at devoxx belgium by cedric from the build tool team not improving android field performance um there's also this talk on gradle for large projects android projects i also gave this talk on scaling android builds where i give some more of my recommendations for making entry builds fast there's also this talk on this my gradle doctor plugin that i did at virtual android makers earlier this year um and then there's also this blog post on measuring build cache performance and then yes i mentioned this earlier but i recommend watching this talk by nokia villar and then there's also a panel on android builds at scale tomorrow which i also recommend coming to i'll be there as well as zach chuki israel john and ivan and that's it uh so thank you for this talk i'm about to open up for questions if you're interested in a grail enterprise trial you can click that link uh we're also hiring the gradle solutions team we we could use some help um and then there's also a link to the slides at the bottom of the build so yeah thanks everyone for listening i hope that was helpful no matter if you just have a you know and no matter what the experience level in in gradle of course feel free to contact me if you have any more questions in any of that and uh yeah now i will open it up for questions

okay first question is the one second versus 10 millisecond difference in between the groovy versus build scripts fixed or does it go for bigger scripts this is not fixed i mean it's fixed it's very very significantly improved so i think build script compilation depending on your project is going to be about 10 times faster for kts scripts and it's going to use less memory you can take a look at the release notes for the gradle 6.8 milestone three and uh in that it'll tell you just how much faster these performance improvements are yeah next one g-way i don't think my company wants to spend money on greater land price but it's possible just to pay for an expert to examine our build um you can ask so there's twofold answer to that question one is i think gradle enterprise is beneficial to every company no matter how small or large you are and we make the business case for purchasing gradle enterprise and i'll say that in almost all cases you actually save more money by purchasing gradle enterprise based on all the money and time you save with gradle enterprise and that's based on calculations of how much each developer minute is worth at your company and you can use the gradle enterprise roi calculator on the website uh to calculate that now okay so you might not believe it or not whatever but you can use a free trial of gradle enterprise and see if it's actually worth it for your build so that's going to be the answer for of me tooting my own horn but if it's if you need more advice on your gradle build feel free to use the gradle community slack i should have linked to that in my presentation but just google gradle community slack it's open for anyone uh next one z-way our build time went from two minutes to five minutes over the past two years would you recommend modulize inside a project for making components external libraries yes i really recommend modularizing your build especially because configuration caching will make it so that no matter how many modules you have it doesn't really significantly impact the configuration phase anymore modularization is is really the way to fix that now of course it could be other things wrong with the build but yeah modulation should help a lot uh chao zhang how fast is gradle going to remove the incubating annotation my company has been investing a lot to trying out new features to boost our build uh this is a really good question i i would say in my opinion uh the incubating annotation is not always really incubating a lot of the things that are actually incubating are just really stable um but if if there's a specific example maybe that's

worth asking more but if it works for your build it works for your build right um i guess it depends on what you're concerned about yeah but good questions

all right if that's all the questions um thanks so much everybody it's really great that you guys all came to my talk and thanks yeah you