Profile Your Lint Tasks

I recently read a post by Tor Norbye about increasing Lint performance. I also watches a talk by Aurimas Liutikas about profiling your app builds. I thought I would combine these two ideas to try to profile my lint tasks.

In his post, Tor advises to only run lint on app/leaf modules:

If you have divided your project into many smaller modules – a number of libraries and just a couple of app modules, it’s much better to (a) turn on checkDependencies mode and (b) only run lint on the app modules, instead of recursively running lint on each module.

Add a checkDependencies flag to your app/leaf modules and then only run lint on that one module ./gradlew :app:lintDebug

android {
    ...
    lintOptions {
        ...
        checkDependencies true
        ...
    }
}

Recently, after reading Tor’s suggestion, I switched from running lint on every module to running on just leaf modules. I just assumed that it would make my lint tasks faster. I didn’t really think anything more about it. Later I reread the whole conversation on Tor’s post and I saw that another developer had actually run some benchmarks and got unexpected result. This got me thinking. They were right to assume Gradle’s parallelization should be faster. That’s when I realized I should actually profile lint, which in turn got me thinking about Aurimas’ talk about profiling builds in general. I realized that I could easily profile these two lint tasks and see for my project which was faster.

The Experiment

I’d profile both options: parallel + run on each vs. checkDependencies = true and compare build times. Each test would begin w/ resetting the Gradle Daemon and a clean

$ ./gradlew --stop && ./gradlew clean

For the first test I enabled checkDependencies in the app module and profiled app:lintDebug via:

$ ./gradlew --profile --offline --rerun-tasks app:lintDebug

After three runs (resetting daemon + clean each time) I got

| test | duration (sec) |
|------|----------------|
|  1   |     127.6     |
|  2   |     118.6     |
|  3   |     105.2     |
|  4   |     99.7      |
| avg  |     112.8     |

For the second test I repeated the process but ran lint tasks per-module in parallel. I removed the checkDependencies and ran lint on all my modules in parallel:

$ ./gradlew --profile --offline --parallel --rerun-tasks\ 
    app:lintDebug\
    lib1:lintDebug\
    lib2:lintDebug\
    ...
    lib7:lintDebug

The results from this were

| test | duration (sec) |
|------|----------------|
|  1   |     90.6      |
|  2   |     84.0      |
|  3   |     94.5      |
|  4   |     84.0      |
| avg  |     88.2      |

I saw a difference of ~24 seconds, a 20% improvement by using the per-module approach, instead of the leaf-only approach.

A Tale of Two Projects

So why did I see these numbers which seam to go counter to what Tor is recommending? To understand the difference we have to understand how my project is different from the projects Tor is probably bench marking against. (These assumptions about his projects could be wildly inaccurate, I’m just guessing.)

My project is eight modules: two leaf “app” modules and six Java/Android library modules. When I deliver my apps I build/test/lint each app module. I never release one app and not the other. So for me if I ran lint on just the leaf modules using checkDependencies I’d end up duplicating a lot of the linting on library modules. Note: In the above tests I removed linting on one of the app modules for both test cases to avoid this exact duplication. So my benefits would be even greater when I use parallelization.

In Tor’s cause I bet he is drawing a lot from the AndroidX and Android Studio projects. Earlier in his talk Aurimus mentions that AndroidX is now 240 Gradle modules. This comprises dozens of leaf modules (e.g. Live Data, Navigation, Room, CameraX) in addition to hundreds of internal or common lib modules. As such, I suspect that the total dependency tree of AndroidX is a lot wider and flater that most Android apps. Each leaf module might depend on only a few of those 240 modules. So if you are a Google developer opening a PR for changes to CameraX, it’s unlikely that you’ve changed code in many modules. You likely don’t care if lint passes across the entire project, only for CameraX and its dependencies. Running lint for all 240 modules would surely be a waste.

Considerations

There are a few things to consider when running these benchmarks yourself or drawing any hard conclusions about my own data. You have to run these profiles on your own project to really know for sure. No two projects will be the same. The goal here is to know with some certainty which approach is fastest.

You have to consider your use case. Are you linting everything at once? Are you only linting as you release? In my case, I run lint on all the things with every master merge or open pull request. So it definitely makes sense for me to use the per-module approach. As I said I deliver both apps at the same time so it makes sense not to duplicate running lint on all the lib modules.

Your baseline may need to change. If you run lint for each module you’ll need a baseline file for each module. If you run lint for just the leaf modules you’ll have baselines just for those, which might duplicate any suppressions for lib modules.

One benefit of running lint on just the leaf modules is that lint will combine all errors into a single HTML report. Otherwise you’ll have an HTML report for each module. For me this is not a big deal because I rarely have any new errors across multiple modules. I usually only have one HTML report that needs my attention.

You also need to consider if all of your modules maintain the same min / compile Sdk. If they have different values, you may want to consider a leaf-only approach. As Tor writes:

Many errors take into consideration factors like the minSdkVersion, the compileSdkVersion, etc. These are typically defined in the app module. A library can have a lower minSdkVersion than the app module, so when performing analysis you really want to apply the effective API level. In global analysis, that’s what lint looks for (detectors are passed in not just local information, but a reference to the main project as well (typically the app project), and lint rules which look up the minSdkVersion pretty much always look up the main minSdkVersion, not the local one.

I’m not entirely certain that running lint on just a leaf module with checkDepencencies true and checkTestSources true will actually lint all test sources in the non-leaf modules. It may only run lint on test code which is accessed from the leaf’s test code.

Lastly, this discussion has no bearing on running lint for all buildTypes. Regardless of which method you take here, running lint on only release (or only debug) version will speed up lint. So be sure you are always calling lintDebug or lintRelease, never lint.

Update: As of AGP 7.0.0. lint will run on the default variant only. You don’t have to worry about specifying a variant.

I hope this helps determine how you should run lint on your project. I the end you should always profile before taking any effort to optimize.