Help me profile Elm 0.19.2 compiler speed!

Request Feedback
1

Context

I have figured out a few compiler optimizations that I would like to implement in Elm 0.19.2. I want to profile the improvements on large amounts of code, so to get a baseline, I am synthesizing Elm projects with the following results with 0.19.1 on my laptop:

| Lines  | Time  | Modules | Incremental |
|--------|-------|---------|-------------|
| 105827 | 0.87s |  256    |             |
| 212067 | 1.50s |  512    |             |
| 424547 | 3.02s | 1024    |             |
| 849507 | 6.41s | 2048    |       1.32s |

It’s synthetic code, so this is just a baseline! This is the starting point to improve upon.

Request

I would like to compare against some common JS/TS setup, but I need help! Can you share a setup for a JS or TS project with a single main file? I can set up the equivalent synthetic code from there. I just need help knowing what a JS/TS developer would consider a “normal” development setup. (bun, tsc, jsx, etc. etc. What is typical? Do you have a bash script that’ll set it up reliably?)

24 Likes
2

I just need help knowing what a JS/TS developer would consider a “normal” development setup

I can’t share anything, unfortunatly, but most projects I’ve worked on lately has used Vite ( Getting Started | Vite )

I think it’s worth mentioning why Vite is heavily used, though.

Vite relies on native JS modules and delivers them just in time. So when you point your browser at `localhost:1234` or whatever, you’ll see one http get request for every .js module that that page ends up using.

This means that vite is very fast because it doesn’t actually compile anything.

This also happens with typescript. In the projects I’ve worked on, vite just strips away the type definitions to be able to deliver the js. Type checking is done as a separate step, usually by the IDE or in a pre-commit hook.

You can do production builds, of course, which will compile and minify the project. It’s usually only done as part of an automated deploy, not by the developers themselves.

3 Likes
3

Hi Evan! Exciting to hear about a new Elm version!

I’ve a couple questions RE useful test data:

  • There’s quite a number of different ways to approach frontend projects, does it matter the specifics? For example, should we be comparing bundling to a single file? Or would projects that perform code splitting or no bundling at all be appropriate?
  • Similarly, should minification and compression be part of it?
  • What about web frontend languages? I have a benchmark of a synthetic Gleam v1.14 project.

Cheers!

edit:

It’s a bit of a tricky one, as what is normal will vary rather a lot, with each community having different preferred approaches and expectations. Perhaps a good approach would be to look at download counts on npm for various tools, and then compare against the ones with the highest recent download count. Though this will exclude the many communities that use tools from other sources, unfortuntely.

1 Like
4

I agree that Vite is the default first choice for building new JS projects. npm create vite@latest=> vanilla => TypeScript will give you a very straightforward template with a single main file that you can flesh out however you’d like.

I would also emphasize Robin’s point above about how Vite works. I’d suggest that comparing Vite build times to Elm build times will inevitably be a bit of an “apples to smoked salmon” comparison. Vite has two basic modes: vite, which starts a dev server, and vite build, which is intended for production builds.

If the scaling factor is “Lines in the project,” vite has O(1) clean build times, and O(1) incremental build times. In reality of course this just means that the build time is amortized over the life of the developer’s session: as Robin describes, vite is going to transform each ES module as it’s needed in the browser during the session.

vite build will use a completely different toolchain to produce a production build. This could be benchmarked against Elm as an end-to-end process, but again it’s not apples-to-apples because the supported build output is so different: an 800k-line JS project is likely going to be split into hundreds of small JS files that can be lazy-loaded, whereas an 800k-line Elm project is going to emit a single JS file. (I am assuming here that Elm 0.19.2 does not include the ability to output; if it does, you can discount this part.)

And of course, distinct from either of these will be the amount of time the TypeScript language server takes to type check and send information to the user’s editor, which is where most of the checking process happens.

1 Like
5

Hey! Excited to hear more about what is coming! :slight_smile:

To add a bit to what everyone already said. I looked for a couple of non-trivial open source applications built with React. I think using these would be the easiest way to do a real world benchmark!

Plane is a project management tool built with React + Turborepo (Vite as bundler).

Excalidraw is a collaborative white board tool built with React + Vite.

You can disable the default code splitting by passing the proper options to rollup - used by vite for production builds.

Though, as people mentioned, dynamic module imports is currently a widely available platform feature so deciding not to do code splitting is quite unusual these days.

2 Likes