Slow Disks - Caching fileHashes 10 Min Faster

Slow Disks - Caching fileHashes 10 Min Faster

I recently discovered that my CI builds were crazy slow because of slow Disk I/O.

I painstakingly experimented with everything that made the second build so much faster than our first build on CI.

And I knew that the solution was to get an agent with local SSD, but I saw how much fileHashes was contributing and I figured that I would experiment caching fileHashes to see if it made cheaper slow agents more viable.

A Gradle build gets two kind of fileHashes. project level and user level

In order for Gradle to reuse them, your files must have the same mtimes as the files in your project. However the typical git checkout will reset the mtimes on all of your files, so being able to reuse fileHashes in a CI setting required using a tool called git-restore-mtime to look through git logs and restore mtimes.

Interestingly, I found that I only had to populate cache with restored mtimes, and then subsequent builds could use fileHashes without having their own mtimes restored. But if I populate cache without first restoring mtimes, then it wouldn't work.

Long story short, with our crazy slow File I/O on CI, it saved 10 minutes

I tried this again on an experiment with the i7i and it was 20s faster, but it could have been statistical noise, so I am abandoning this for now as I focus on things with more impact.