Some advice on Elm compile performance

MarkHamburg 2018-02-01 15:43:33 UTC #1

I spent some time yesterday digging into Elm compile performance and discovered a few interesting things:

elm-make is exceedingly naive compared to many (most?) build tools with respect to dirtiness propagation. Make a no-op change to a module and everything that imports it directly or indirectly recompiles. This propagation should only be needed if the exported interface changes. The net effect of this is to make even small changes more expensive as programs grow bigger and to discourage the sort of hierarchical decomposition and reuse that has been standard software engineering practice for decades. From a practical standpoint, this issue argues for using CSS as much as possible to style interfaces so that styling tweaks don’t need to spark Elm rebuilds.
Something one can act on when building modules: Try to avoid exporting records that are essentially private. If the model defined in your module is a record with a bunch of fields only of relevance to the code in your module, wrap that model inside an opaque type. This is because when you export the record, all the things it contains becomes part of the interface for your module and that leads to huge elmi files which slow compilation. Weirdly, the size benefits of doing this wrapping only seem to kick in a bit further up the hierarchy, but wrapping a few cases in our codebase roughly doubled our build speed.

Mark

brian 2018-02-01 16:03:47 UTC #2

Based on this, I suspect the benefit has more to do with the module organization than it does with the size of the exports. Other people have seen similar things. But it does sound like the two are interacting in a weird way, which is good to keep in mind!

I imagine you may have seen this, but @ilias has a nice gist with more details: https://gist.github.com/zwilias/7ed394ec0e9c6035e1874d19b721e294#many-imports-interlinked-module-structure

What were you doing that lead you to do these investigations? Some big slowdowns in compiling your app?

MarkHamburg 2018-02-01 16:10:31 UTC #3

Over a year plus of code accumulation, our app had been getting progressively slower to compile to the point where it had gone from “yeah, it always seems to get stuck for a while at 69%” to “this is really annoying and a productivity killer”.

Mark

MarkHamburg 2018-02-01 16:15:07 UTC #4

The size of the exports issue came up because I was seeing interface files with sizes in the tens of megabytes. Just by wrapping some records in opaque types and without changing module organization, I dropped those sizes down dramatically and sped compile times dramatically. The remark about the sizes still being a bit difficult to interpret stems from the fact that it isn’t the elmi of the module where I did the wrapping that got smaller but rather the elmi of the module importing it. I guess this could be explained by assuming that even for opaque types the elmi includes everything visible within the module and leaves it to importing modules to not use that data.

Mark