commit 054e84a47f9fa51764a13e3775b86299bc75521b Author: Stu Hood Date: Fri May 8 14:12:40 2015 -0700 Zinc patches to improve roundtrip time _(applies atop https://rbcommons.com/s/twitter/r/2177/)_ This is an import of two changes we made to improve the roundtrip time from pants to zinc during an incremental compile with the isolated strategy (and parallelism, although that's orthogonal.) The worst case that we've found for incremental compile is many unchanged targets downstream of a changed target. In that case, zinc is invoked for each downstream target, but usually just needs to do enough inspection of inputs to decide that it doesn't need to compile anything. With a graph of 900+ upstream targets, detecting upstream changes was initially filesystem bound. We did a few things to fix this. First, we determined that we weren't actually [providing all relevant analysis to zinc](https://rbcommons.com/s/twitter/r/2155/), so it was recomputing in some cases. After fixing that, the next two hot things in the profile were: 1. In order to hit the in-memory analysis cache, zinc uses a SHA1 hash of upstream analysis files in order to decide that it could use them. This was admirably thorough, but almost certainly overkill. Instead, we use the analysis filename and timestamp here. It would be nice to upstream a patch to include a UUID or something in the header of each analysis file to make this bullet proof though. 2. To decide where classes have been defined, sbt uses a `definesClass` function that (for a given directory/jar) returns a lookup table from String=>Boolean which indicates whether the relevant directory/jar defines the class. Oddly enough, it was not using the upstream analysis to decide whether directories contained particular classes, and was instead stat'ing files and listing jars in all cases. So, instead, when we have valid upstream analysis this patch uses it to execute class lookups. ---- Together, these two changes took the roundtrip time for a noop zinc compile with 900+ upstream analysis files from 5+ seconds to <0.5 seconds. Testing Done: deployed internally for java and scala. could definitely use tests though. Reviewed at https://rbcommons.com/s/twitter/r/2178/ src/scala/org/pantsbuild/zinc/Compiler.scala | 13 ++++++--- src/scala/org/pantsbuild/zinc/FileFPrint.scala | 35 ++++++++++++++++-------- src/scala/org/pantsbuild/zinc/Inputs.scala | 38 ++++++++++++++++++++++---- src/scala/org/pantsbuild/zinc/Main.scala | 2 +- 4 files changed, 65 insertions(+), 23 deletions(-) commit 882e709527bd8421561dd0ca12f30530e9899d88 Author: Stu Hood Date: Fri May 8 09:58:58 2015 -0700 Revert to zinc 0.3.7 with only package-name edits This change reverts the imported zinc to exactly [v0.3.7](https://github.com/typesafehub/zinc/tree/v0.3.7) with only edits to the package name. We have a few patches to zinc that we'd like to incorporate here, and having patches apply with only minor edits makes life much easier. Additionally, it would be nice to allow upstream to consume patches we apply if it begins to liven up over there. Testing Done: ./pants test zinc/:: src/scala/org/pantsbuild/zinc/:: Reviewed at https://rbcommons.com/s/twitter/r/2177/ src/scala/org/pantsbuild/zinc/Compiler.scala | 169 +++++++-------- src/scala/org/pantsbuild/zinc/Inputs.scala | 267 ++++++++++++------------ src/scala/org/pantsbuild/zinc/Main.scala | 8 +- src/scala/org/pantsbuild/zinc/SbtAnalysis.scala | 16 +- src/scala/org/pantsbuild/zinc/Settings.scala | 58 ++--- src/scala/org/pantsbuild/zinc/Setup.scala | 25 ++- src/scala/org/pantsbuild/zinc/Util.scala | 172 ++++++++------- src/scala/org/pantsbuild/zinc/ZincClient.scala | 42 ++-- 8 files changed, 373 insertions(+), 384 deletions(-) commit 674c205de89ac19063c8c7286cf861b866eb4ee9 Author: Tom Howland Date: Wed Apr 22 20:50:48 2015 -0700 Import zinc to the pants repo, and drop its sbt build. src/scala/org/pantsbuild/zinc/Cache.scala | 93 ++++++ src/scala/org/pantsbuild/zinc/Compiler.scala | 220 ++++++++++++++ src/scala/org/pantsbuild/zinc/FileFPrint.scala | 39 +++ src/scala/org/pantsbuild/zinc/Inputs.scala | 244 ++++++++++++++++ src/scala/org/pantsbuild/zinc/Main.scala | 117 ++++++++ src/scala/org/pantsbuild/zinc/Nailgun.scala | 138 +++++++++ src/scala/org/pantsbuild/zinc/Options.scala | 272 ++++++++++++++++++ src/scala/org/pantsbuild/zinc/SbtAnalysis.scala | 286 +++++++++++++++++++ src/scala/org/pantsbuild/zinc/Settings.scala | 362 ++++++++++++++++++++++++ src/scala/org/pantsbuild/zinc/Setup.scala | 315 +++++++++++++++++++++ src/scala/org/pantsbuild/zinc/Util.scala | 289 +++++++++++++++++++ src/scala/org/pantsbuild/zinc/ZincClient.scala | 216 ++++++++++++++ 12 files changed, 2591 insertions(+)