Optimizing compile times?

[jarcane]

I’m currently working on a very large internal project doing some refactoring and code cleanup, a lot of which involves repeated recompilation.

The problem is I’m having serious issues with slow build times in the 2 minute range pretty consistently, which makes things quite a lot slower and more difficult than they ought to be.

I suspect part of the culprit may be some changes in a previous update, and I’ve managed to dig up some stuff about a performance bug with the exhaustiveness checker on tuple matches, but other than “wait until 0.19 fixes it”, are there any other things I should look at to improve the situation? Is it possible even to profile a compile itself, so I can at least find where the slowdown is happening?

[folkertdev]

you should ask in the #compile-time slack channel, there are some very knowledgeable people there.

I think the other big problem right now is many interconnected modules. The elm-analyse tool
can visualize your module dependency graph to help you fix that problem.

For profiling I think the only way at the moment is to compile individual files, and check how long they take.

[jarcane]

elm-analyse is actually what I’ve been using to check for some of the other code issues. I did actually fire up the .dot file in GraphViz and it looked like Cthulhu’s spaghetti dinner so if that’s a performance hit then I’m gonna guess that’s a big one. Odd that it should hit only recently though. Most of the interdependency issues are long-standing.

[michaeljones]

You might find elm-module-graph: https://github.com/justinmimbs/elm-module-graph to be useful as well. I’ve not seen the elm-analyse visualisation but the module-graph one is quite understandable.

I have found it very useful in the past for figuring out why something was being included when I didn’t expect it to be.

[Sebastian]

Someone suggested me to check the compiled files timestamps to see how long they take to compile.

I compiled our app by forcing only one core as discussed here https://github.com/elm-lang/elm-compiler/issues/1473.

Then started comparing timestamps for .elmi files (made a script that will find the bigger differences).

Unfortunately I found nothing useful by doing this exercise, I had trivial modules showing a long time to compile according to this method.

[turboMaCk]

There were several useful tips already mentioned (I didn’t know about the slack channel, thanks!) so I’ll add just my own habits.

background skip if you’re not interested
We also have big single-page app. Just compile times of the whole app on our CI server takes over 3 minutes which is similar what most of my colleagues experience on laptops as well. I have MBP 15" myself and overkill Arch Linux work station I mostly prefer to use myself. I’m also Emacs user so some of my habits are influenced by Emacs ways of doing things.

works for me:

• If you have a large number of threads on your system (I have 8 core 16 thread machine) use sysconfcpus trick even on your system. I get best performance using just single-thread of elm-make which is about 5times faster than compiling on 16 threads.
• The biggest problems with incremental compiles (during development) are changes in shared modules. If you make a change in module all modules which import it has to be recompiled as well. If I’m working on such module I’m usually compiling just that that one (by loading it to repl). Sometimes I even rather pushing just function after function into repl.
• When I’m working on a module that is just part of the app (not shared) I’m using watcher since compile times are good enough.
• If neither of those is enough than I start thinking about changing code in a way it compiles more quickly. Usually, this doesn’t happen that much though (might be still necessary if your app is over 100k LOC though, our isn’t on half of that yet so I have no experience with large-large).

Using repl has its flaws though. It can’t read type definitions so if I’m doing more type driven parts of API design where I’m replacing implementation with Debug.crash I can’t use push to repl. Loading module to repl still works though.