Improving Compilation time: Insights from Elm Compiler Internals

Show and Tell
1

I found an old topic in 2018 about compilation times. According to it, changing large records to opaque types can dramatically improve the situation, and when we tried it in the SPA application we’re building at our company, it actually decreases compilation time and memory usage by 20~30% (from about 80sec to 60 sec).
Some advice on Elm compile performance

To investigate the reason, I explored the compiler’s source code and discovered that there is an issue with how type aliases are handled internally.

For example,

type alias Three a = {p: a, q: a, r: a}

var: Three Int

When dumping the type of the above var from the compiler, it looks like this.

  (TAlias 
    (Canonical {_package = Name {_author = author, _project = project}, _module = Main})
    Three 
    [(a,TType (Canonical {_package = Name {_author = elm, _project = core}, _module = Basics}) Int [])]
    (Filled (TRecord (fromList [(p,FieldType 0 (TType (Canonical {_package = Name {_author = elm, _project = core}, _module = Basics}) Int [])),(q,FieldType 0 (TType (Canonical {_package = Name {_author = elm, _project = core}, _module = Basics}) Int [])),(r,FieldType 0 (TType (Canonical {_package = Name {_author = elm, _project = core}, _module = Basics}) Int []))]) Nothing))
  )

Simplified by removing the syntactic noise, it looks like this.

TAlias (author.project.Main.Three)
  [(a, elm.core.Basics.Int[])]
  (Record 
   [(p, elm.core.Basics.Int[])
   ,(q, elm.core.Basics.Int[])
   ,(r, elm.core.Basics.Int[])
   ])

Each time you pass a type argument to an alias, its contents are expanded (usage count + 1) times. In this case, it’s just an Int, so it’s fine, but if you pass a large record, it becomes quite problematic.
This type information is expanded each time the type is used and is binary-encoded in .elmi files within elm-stuff. When I passed the -p option to elm-compiler to take a profile, 90% of our project’s compile time was spent parsing elmi files.

My next goal is to work on improving the encoding of .elmi files, but for now, using opaque types for large records is a practical and effective solution.

14 Likes
2

Hope this helps. @kanishka

3

GJ on the writeup!

One question, in this case it was a type alias (record) with type variable, but does the situation also applies to ordinary record without type varibales?

1 Like
4

Thank you for your feedback. Very good point.

The situation indeed apply to ordinary records without type variables as well. When record fields are expanded in the .elmi files, it can become problematic if the record has many fields with large types.

1 Like
5

As additional information, I’ll share a convenient method for finding large records.

Navigate to the elm-stuff/0.19.1 directory and execute du -hs * | sort -h | tail to find the large .elmi files.
It’s worth to note that the .elmi file for the module where the large record is defined might not be the largest itself, as type aliases are expanded each time they are used in type definitions. Find the large record used in the module with large .elmi file and consider making the record opaque type, or do the same on the frequently used types in the record fields.

6

Thank you!!! I’ll try this on some larger apps and report back when I have some down time.

1 Like
7

I’m rather interested in cases where this hack doesn’t work. I’d love to hear the results anyway.

1 Like
8

Hi! Thanks for the write up, we’re also experiencing some slowdowns in our app, this will be useful insights!

What is this -p option? I tried elm make src/Main.elm -p but the compiler doesn’t recognize it. Did you tweak the haskell code?

1 Like
9

Ah, I should have written more about it.

To determine which functions are consuming resources in a Haskell program, you have to build the program with profiling option by following instruction in the link, and run it with something like
~/compiler/dist-newstyle/build/x86_64-osx/ghc-8.6.5/elm-0.19.1/x/elm/build/elm/elm make +RTS -p -RTS --output /tmp/main.js src/Main.elm .

It’s a bit cumbersome, but this process gives you all about the performance.

1 Like
10

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.