Our Experience upgrading to 0.19 at Humio


#1

Hello Elm Community!

Where I work (Humio), we have a really big and old Elm code base, and just a few weeks ago we upgraded it to Elm 0.19. I’d like to document how that went for us (short answer, it went well :tada: ).

About the code base

How old is it

Its about 3 years old, and goes back to Elm 0.15. Parts of our code base has actually remained unchanged since those ancient Elm versions, and if you poked around you would see names like Action and State.

How is it structured

Its almost all Elm (>80,000 lines of Elm). Its just one big Elm SPA plus some web components and ports.

How did we upgrade

First things first, we had to migrate a lot of native code. A lot of it was just migrating our chart UIs into web components. Other parts, like deleting our local storage native code forced us to reconsider how we were doing local storage entirely, which was probably for the best anyway.

After that we switched over to the 0.19 compiler, and just took the compiler errors one at a time. I would say this took 2.5 days, and 90% of this work was just tedious changes. A huge chunk of it was either changing (a, b, c, d) to (a, b, (c, d)), changing Float time to Posix time, or changing things to the correct toString function.

Once we got everything compiled it mostly just worked. I can’t really recall any functional problems with the result. We did discover after sharing the code around our office for people to test, that browser extensions and third party adds on interfered with the virtual Dom, due to Browser.application taking over the whole <body/> (see thread here). Resolving that was pretty straight forward tho, we just added some ugly scripts to the build process that modified the compiled JS

Overall

I would estimate that between doing pre-emptive refactoring, deleting native code, handling compiler errors, changing our build process and then testing, it took us a total of 2.2 weeks of developer time to migrate from 0.18 to 0.19

Results

Methods

For the Elm 0.18 times below, I would first delete elm-stuff, then install the packages using elm-package install. After that I would time how long it took to build to a js file. For the 0.19 numbers it was the same, except for the fact that I would not install the packages before hand (since Elm 0.19 will pull them out from a cache and Elm 0.18 downloads them, I think this might bias the results against the 0.19 numbers below, since they include the time it takes the elm compiler to copy the packages from the cache). For the bundle size I would uglify and gzip the output.

0.18

81,838 lines of Elm

Results of three compiles:

real 77,93s user 17.10s system 130% cpu 1:12,98 total
real 79,52s user 18,48s system 133% cpu 1:13,68 total
real 78,38s user 16,48s system 130% cpu 1:12,92 total

Size of compiled js (uglified, gzipped) = 336,8 KB

0.19

81,791 lines of Elm
Results of 3 compiles

real 2,73s user 0,37s system 82% cpu 3,782 total
real 2,75s user 0,37s system 81% cpu 3,830 total
real 2,76s user 0,39s system 79% cpu 3,973 total

Size of compiled js (uglified, gzipped) = 319 KB

Size of compiled js in optimized mode (uglified, gzipped) = 282,5 KB

Comparison

The average real time of the 3 installs in Elm 0.18 is 78,61s

The average real time of the 3 install in Elm 0.19 is 2,74s

Whats a good way to express those gains? 78,61 / 2,74 = 28.68, so a x28.68 increase in our compile speed and a (336,8 - 282,5) / 336,8 = 16.1% decrease in bundle size, right? Pretty good!

EDIT: I got the compression size numbers wrong and had to fix them in this post, they are accurate now.
EDIT 2: Our codebase goes back to Elm 0.15, not Elm 0.16


#2

Love this hack :sunglasses:


#3

Thanks for the detailed post…

Can you point out any libraries you used for data visualization? In the JS world, D3 and Highcharts seem to be the most-used libraries. Did you bind to those?

Did you have any use for websockets in your product?


#4

We use highcharts, but I think we arent happy with it. In particular its not very reactive, so our web component has to do ugly stuff like storing parameters, checking for differences, and then forcing high charts to re-render with those differences. I think we would prefer vega today, but its just a question of if and when it worth it to migrate.

We dont.


#5

Anyone considering alternatives for high quality reactive data visualization in Elm may wish to have a look at elm-vega and elm-vegalite. It is a binding to their respective javascript libraries, but handled efficiently through ports. Vega is close to D3 in the degree of flexibility offered. Vega-Lite is good for creating from a more standard suite of charts and plots with relatively little Elm code.


#6

@jwoLondon we are migrating to Vega, and have some vitualizations already using it. Not sure whether elm-vega or elm-vegalite brings much to the table compared to making our own tailored solution but I will give them a serious look. I saw them a while back, but maybe lots has changed - thanks for the suggestion!


#7

You should totally check out Elm-Visualization

It’s basically a D3 replacement written in Elm. We use it for all sort of things! (Fancy charts, Donut Charts, etc.) Also works great with TypedSVG:

If you have the time, way better than Highcharts, etc. as you can really customize, also it’s reactive properly.