I have figured out a few compiler optimizations that I would like to implement in Elm 0.19.2. I want to profile the improvements on large amounts of code, so to get a baseline, I am synthesizing Elm projects with the following results with 0.19.1 on my laptop:
It’s synthetic code, so this is just a baseline! This is the starting point to improve upon.
Request
I would like to compare against some common JS/TS setup, but I need help! Can you share a setup for a JS or TS project with a single main file? I can set up the equivalent synthetic code from there. I just need help knowing what a JS/TS developer would consider a “normal” development setup. (bun, tsc, jsx, etc. etc. What is typical? Do you have a bash script that’ll set it up reliably?)
I just need help knowing what a JS/TS developer would consider a “normal” development setup
I can’t share anything, unfortunatly, but most projects I’ve worked on lately has used Vite ( Getting Started | Vite )
I think it’s worth mentioning why Vite is heavily used, though.
Vite relies on native JS modules and delivers them just in time. So when you point your browser at `localhost:1234` or whatever, you’ll see one http get request for every .js module that that page ends up using.
This means that vite is very fast because it doesn’t actually compile anything.
This also happens with typescript. In the projects I’ve worked on, vite just strips away the type definitions to be able to deliver the js. Type checking is done as a separate step, usually by the IDE or in a pre-commit hook.
You can do production builds, of course, which will compile and minify the project. It’s usually only done as part of an automated deploy, not by the developers themselves.
Hi Evan! Exciting to hear about a new Elm version!
I’ve a couple questions RE useful test data:
There’s quite a number of different ways to approach frontend projects, does it matter the specifics? For example, should we be comparing bundling to a single file? Or would projects that perform code splitting or no bundling at all be appropriate?
Similarly, should minification and compression be part of it?
It’s a bit of a tricky one, as what is normal will vary rather a lot, with each community having different preferred approaches and expectations. Perhaps a good approach would be to look at download counts on npm for various tools, and then compare against the ones with the highest recent download count. Though this will exclude the many communities that use tools from other sources, unfortuntely.
I agree that Vite is the default first choice for building new JS projects. npm create vite@latest=> vanilla => TypeScript will give you a very straightforward template with a single main file that you can flesh out however you’d like.
I would also emphasize Robin’s point above about how Vite works. I’d suggest that comparing Vite build times to Elm build times will inevitably be a bit of an “apples to smoked salmon” comparison. Vite has two basic modes: vite, which starts a dev server, and vite build, which is intended for production builds.
If the scaling factor is “Lines in the project,” vite has O(1) clean build times, and O(1) incremental build times. In reality of course this just means that the build time is amortized over the life of the developer’s session: as Robin describes, vite is going to transform each ES module as it’s needed in the browser during the session.
vite build will use a completely different toolchain to produce a production build. This could be benchmarked against Elm as an end-to-end process, but again it’s not apples-to-apples because the supported build output is so different: an 800k-line JS project is likely going to be split into hundreds of small JS files that can be lazy-loaded, whereas an 800k-line Elm project is going to emit a single JS file. (I am assuming here that Elm 0.19.2 does not include the ability to output; if it does, you can discount this part.)
And of course, distinct from either of these will be the amount of time the TypeScript language server takes to type check and send information to the user’s editor, which is where most of the checking process happens.
To add a bit to what everyone already said. I looked for a couple of non-trivial open source applications built with React. I think using these would be the easiest way to do a real world benchmark!
Plane is a project management tool built with React + Turborepo (Vite as bundler).
Excalidraw is a collaborative white board tool built with React + Vite.
You can disable the default code splitting by passing the proper options to rollup - used by vite for production builds.
Though, as people mentioned, dynamic module imports is currently a widely available platform feature so deciding not to do code splitting is quite unusual these days.
We migrated from Webpack a few years ago. The DX is actually much worse with Vite - full re-load freezes my browser for a few seconds (it downloads ~1700 js files, so that means ~1700 requests / ~30MB). And due to spaghetti architecture in older parts of our codebase, many files are not eligible for hot-reload, and any edit in those files causes full-reload.
But I have old laptop - apparently if you get the newest and most expensive laptop, the full-reloads are fast-enough. Or if you refactor your entire codebase to eliminate as many full-reloads as possible.
Test runner: Vitest with jsdom environment
We test entire React components by rendering them in the jsdom environment. We also do a lot of mocking (and it’s very un-reliable)
Linting: ESLint 9 (flat config)
Prettier - enforced and auto-fixed by ESLint
Stylelint (for SCSS) - enforced and auto-fixed by ESLint
And IDE is set to auto-fix on save, so we get consistent auto-formatting this way.
With typescript-eslint (all the type-aware rules enabled). We try to use the default/recommended rules, but we have a long list of custom overrides anyway.
Styling: Tailwind CSS + SASS/SCSS + PostCSS
new code uses Tailwind only
Key scripts (in package.json):
"start": "vite",
"build": "tsc && vite build",
"check": "yarn run type-check && yarn run stylelint && yarn run lint && yarn run test", // To reliably check the entire project before committing - IDE only shows errors for files that were open
"test": "vitest run",
"test:watch": "vitest",
"format": "prettier --write \"src/**/*.{ts,tsx}\"",
"lint": "eslint . --report-unused-disable-directives --max-warnings 0",
"type-check": "tsc --pretty --noEmit",
(And CI pipeline does all those things too)
As for bash script to set up the setup, I don’t think that would be useful for me. I don’t set up many new projects - I work on the same project for multiple years, and in that time, the technologies change too much. Also every project I worked on has different technical needs (it usually integrates into a larger/older system, with different constraints).
I know the compiler speed is part of a nice developer experience.
And in my experience, the Elm compiler is already quite nice on that aspect too.
What is concern me a bit more, is the improvement that could be made on the runtime. Especially, on the size of the final JS bundle, and on the overhead of some construction.
About the bundle size, the code shipped for a hello world is 4k lines and 87kB. For sure minifiers can help, but as they don’t understand Elm and all its guaranties, such as purity and immutability, they don’t optimize much.
About the overhead, I think about the large usage of foo=F2(f); A2(foo,a,b), valid for n from 2 to 9 - which could be simplified into f(a,b). Noticing that recursive constructions can also be simplified : A2(A3(F5(f),a,b,c),d,e) === f(a,b,c,d,e)
I made a micro benchmark with a simple program. And the overhead is real:
node
bun
qjs
raw
978
649
18271
apply
5117
1285
30651
overhead
5.23x
1.98x
1.68x
I guess node and bun are able to JIT this code whereas qjs is interpreting it.
In any case, only the compiler can know that foo is a of arity 2 and that the call site could be simplified.
This and other optimizations have already been described in elm-optimize-level-2 and might be considered as improvements to the compiler.
About bundle size, I’ve not been able to develop a proper static analysis tool, yet. It is still above my skills. But I made a silly experiment.
I used elm-scripting to compile a simple “hello world” program to a script that can be executed in the terminal.
module Hello exposing (main)
import Html exposing (text)
main = text "hello world"
On this generated code, I tested multiple minifiers: uglify, rolldown, rollup and google closure compiler. All are supposed to perform some sort of dead code elimination or tree shaking. But the result on are quite deceptive. I could elaborate more on the results if it arouses any interest.
So I wrote a python program that removes statements and check if the script is still working. Under the hood, it uses tree-sitter javascript parser to manipulate the code.
It targets the elm app code (hardcoded path), then it iterates over all statements.
For each statement, it remove it from the code, then run the code using QuickJS to see if it fails or succeed. If it fails, it restores the statement, else it keep the removal, then moves to the next.
There are 509 statements in the sample app. It would not be possible to test all the combinations (2^509), so statements are iterated in the order.
I get much better result when going bottom to top than top to bottom since statements tend to depends on other statements located above them not bellow them.
Here are the results:
LoC
size (B)
time (ms)
before
4188
85262
20
after
138
2843
4.6
factor
30x
30x
4.37x ± 0.53
Of course, this is a contrived example. Real Elm applications use a much larger portion of the runtime. However, I feel Elm could push its strengths further in terms of bundle size and speed.
I guess everything needed is there. I followed the instructions you gave in the README (clone the repo, npm install, elm make, npm bench), so I have a baseline on my machine. Then I added my results, obtained as explained in my previous message and also results using google-closure-compiler for the record.
Tell me if it needs clarification.
