Understanding Different Gradle Caches for Android Projects, part 3

This is part 3 in a series about different caches available to Android Gradle projects. In part 1 I wrote about the benefits provided by Gradle’s cache of incremental builds and the build cache directory. In part 2 I wrote about Android’s build cache, the Gradle daemon, and dependency caching. Here in part 3, I write about the deprecation and remove of Android’s build cache, and introduce Gradle’s configuration cache.

Deprecated Android build cache

Im my previous post I mentioned Android’s build cache as a separate build cache that sits on top of Gradle’s own build cache. At the time of writing it was unclear what that cache contained. Since then, in August 2020, the Android team has deprecated the cache in AGP version 4.1.0. They eventually removed it entirely in version 7.0, which was released in July 2021.

The AGP build cache was removed in AGP 4.1. Previously introduced in AGP 2.3 to complement the Gradle build cache, the AGP build cache was superseded entirely by the Gradle build cache in AGP 4.1. This change does not impact build time.

The cleanBuildCache task and the android.enableBuildCache and android.buildCacheDir properties are deprecated and will be removed in AGP 7.0. The android.enableBuildCache property currently has no effect, while the android.buildCacheDir property and the cleanBuildCache task will be functional until AGP 7.0 for deleting any existing AGP build cache contents.

This simplifies our understanding of the various Gradle-related caches. I applaud the Android team for explicitly removing this cache. It would have been easy (and customary) of them to just let it die on the vine and sit there for all time.

Gradle’s Configuration Cache

Gradle performs a build in three phases: initialization, configuration, and execution. In my first post, I introduced the “project level” cache and the build cache. Both of these caches preserve output from the execution phase.

The Gradle team, in version 6.6, August 2020, implemented an additional cache for the configuration phase, appropriately named the configuration cache.

Using the configuration cache, Gradle can skip the configuration phase entirely when nothing that affects the build configuration, such as build scripts, has changed. Gradle also applies some performance improvements to task execution as well.

So we may be able to skip the configuration phase entirely, and get a build speed improvement during execution phase.

I should warn that the Gradle team has marked this feature “highly experimental” so you may find additional build-related bugs when using it. However I will address some guidance on when/where to use the configuration cache.

How to use

The easiest way to start using configuration cache is to add the --configuration-cache flag to command line gradle tasks. The cache can help not only with conventional tasks like build/assemble/install but not so obvious tasks like help/check/lint/test.

A note about benchmarking: My sample project has changed quite a lot since my last post. So time comparisons between this post and previous will not be relevant.

Let’s build the project once, without configuration cache enabled, to populate the project-level cache, build cache, daemon cache, and 3rd party dependency cache:

$ ./gradlew app:assembleDebug
BUILD SUCCESSFUL in 38s
98 actionable tasks: 16 executed, 82 up-to-date

Now we can build again with configuration cache:

$ ./gradlew --configuration-cache app:assembleDebug

Configuration cache is an incubating feature.
Calculating task graph as no configuration cache is available for tasks: app:assembleDebug

BUILD SUCCESSFUL in 3s
98 actionable tasks: 98 up-to-date
Configuration cache entry stored.

This drop from 38 seconds to 3 seconds is the improvement from these other caches we’ve already discussed. What is important is that we have now created a reusable configuration cache. So on the third run we can isolate the benefit of the configuration cache:

$ ./gradlew app:assembleDebug --configuration-cache
Configuration cache is an incubating feature.
Reusing configuration cache.

BUILD SUCCESSFUL in 1s
98 actionable tasks: 98 up-to-date
Configuration cache entry reused.

That drop from three seconds to one second is a pretty substantial. It is more meaningful to interpret these results as a 300% improvement, rather than a 2 second improvement. As projects grows in complexity with more modules and plugins, each adding new tasks, the benefits from configuration cache become more apparent.

Increase usage of the configuration cache

Given its experimental nature, I recommend staring to use configuration cache with some of the more common and heavy Gradle tasks: assemble and build.

I prefer using the command line for Gradle work, so I created two aliases to assemble and install with configuration cache:

#alias app_build='./gradlew --configuration-cache :app:assembleDebug'
#alias app_install='./gradlew --configuration-cache :app:installDebug

I’ve seen weird build errors related to configuration cache, especially when changing Gradle build files. Often Gradle will try to reuse an invalid cache and the build gets into a weird state and eventually fails. The easiest step in this case is to re-run the task and ignore the configuration cache, which can be performed by omitting the --configuration-cache flag.

#alias app_build_nocc='./gradlew :app:assembleDebug'
#alias app_install_nocc='./gradlew :app:installDebug

Once you are comfortable that the cache is helping more than it hurts, you can turn it on for your entire project, or your entire build machine by adding to either the system-level or project-level gradle.properties file:

org.gradle.unsafe.configuration-cache=true

You’ll then want the ability to override this flag per-build if the cache is causing errors. You can do so with the aptly named --no-configuration-cache flag:

$ ./gradlew app:assembleDebug --no-configuration-cache

Sometimes, it may help to delete the cache entirely. Unlike the build cache, the configuration cache is project-specific, so you need to remove the directory within your project:

$ rm -rf [PROJECT_ROOT]/.gradle/configuration-cache

I need to do this often enough that I aliased this command as well:

deleteConfigurationCache='rm -rf .gradle/configuration-cache/'

More benchmarking

Let’s run some more benchmarks to see how the configuration cache can help us. Let’s start by making changes to a single Kotlin file within the app module and reassembling:

# change code in a single Kotlin file

$ ./gradlew app:assembleDebug --configuration-cache
Configuration cache is an incubating feature.
Reusing configuration cache.

BUILD SUCCESSFUL in 5s
98 actionable tasks: 5 executed, 93 up-to-date

Our task time increased to five seconds, but the configuration cache was reused. This is awesome but why did that happen? Why didn’t a code change invalidate the configuration cache?

*Understanding the value of the configuration cache requires a deeper understanding of Gradle tasks and how they are organized. I highly recommend reading `Gradle for Android` by Kevin Pelgrims. Understanding the Gradle build system is a deep topic. Kevin extracts most of the Gradle concepts you would need as a professional Android developer and distills them in an easy to read and understand book.(1) Kevin goes into detail about Gradle tasks and their structure. But here we only need to understand that Gradle tasks have a dependency structure: each task can depend on one or more other tasks, acyclicly. If you have a good mental model of the Gradle *module* dependency graph or a Dagger *object* graph, then you can re-use that mental model with Gradle tasks. It’s important to know that the configuration of tasks is *independent* of the execution of tasks.(2) The task graph is computed during the configuration phase. So if a task’s dependencies don’t change then the task graph can be reused. This is a big win for the every-day Android developer since we regularly make code changes and rarely make Gradle task changes. **The configuration cache should give us a boost on almost every run.*

Let’s revert our Kotlin change and assemble again, to get to a fresh starting point. Then reapply the Kotlin change, assemble again with the configuration cache disabled:

# revert change in Kotlin file

$ ./gradlew app:assembleDebug  # get back to fresh state

# reapply the change in Kotlin file

$ ./gradlew app:assembleDebug # now test

BUILD SUCCESSFUL in 6s
98 actionable tasks: 5 executed, 93 up-to-date

We can now compare these two runs:

change	CC enabled	CC used	tasks executed	result
a single Kotlin file	no	no	5	6 sec
a single Kotlin file	yes	yes	5	5 sec

Changing a single Kotlin file requires five tasks to be executed in the execution phase regardless of configuration cache. But if we reused the configuration cache we save a whole second overall.

This one second gain may seem trivial, but in a real project with thousands of Gradle modules and dozens of Gradle plugins, this improvement can be substantial.

If changing Kotlin source files doesn’t invalidate the configuration cache, what does? Let’s make a change to app/build.gradle by arbitrarily bumping the versionCode of our app and reassemble:

$ ./gradlew app:assembleDebug --configuration-cache # fresh start

BUILD SUCCESSFUL in 1s
98 actionable tasks: 98 up-to-date
Configuration cache entry reused.

# change app/build.gradle

$./gradlew app:assembleDebug --configuration-cache
Configuration cache is an incubating feature.
Calculating task graph as configuration cache cannot be reused because file 'app/build.gradle' has changed.

BUILD SUCCESSFUL in 6s
98 actionable tasks: 13 executed, 85 up-to-date
Configuration cache entry stored.

Our staring point here was our one second run, which used configuration cache and build cache. Then changing a single build.gradle file resulted in a time of six seconds. We also see that the configuration cache was not reusable! Some of this six second slow down is because changes to build.gradle cause some of the execution phase to be rerun. We can update our table:

change	CC enabled	CC used	tasks executed	result
a single Kotlin file	no	no	5	6 sec
a single Kotlin file	yes	yes	5	5 sec
a single build.gradle file	yes	no	13	6 sec

Even more benchmarking on a big project

I can repeat these benchmarks on my current project. This is a massive Gradle project with 2,000 modules and dozens of custom plugins. For comparison our sample app which was the basis of our previous tests had ~1300 tasks.(3) My current project has 440,000 tasks.

Building the app with and without the configuration cache has built times of

17 seconds versus 2 minutes 30 seconds.

This is without any code changes. The uncached configuration phase takes over two minutes!

Should you use configuration cache

Yes, absolutely! My advice is to start using configuration cache on a few repetitive tasks like build and assemble. You can start using configuration cache without committing your entire team to it. Eventually move up to an entry in your project-level gradle.properties.

Personally, I don’t find much value yet in using configuration cache on my C.I. builds. For one, I’ve found this leads to a lot of flaky builds which usually required re-running with configuration cache disabled, something not very easy to do on C.I. machines. I also found that configuration cache was rarely the low hanging fruit to improve C.I. build times. If you choose to commit gradle.properties to your repo, for usage by your whole team, I advice you create a tool to swap in separate properties for C.I. builds

Notes

(1) These are my independent opinions. I am not paid to endorse this book.
(2) This isn’t technically correct. I’m simplifying here.

(3) Getting task count:

$ ./gradlew tasks --all --> out.txt
$ wc -l out.txt

A big thanks goes to Martin Marconcini for reviewing this post.