Zokka: Bug fixes for the Elm compiler and core packages

[yosteden]

Hi folks! I’ve already been sharing this on Slack and Discord, but I figured now that I’ve finally wrapped up everything as a nice npm package I’m ready to share this on the Discourse forum. I’ve got a conservative fork of the mainline Elm compiler called Zokka. It aims to preserve complete bidirectional compatibility with the Elm language (i.e. no changes to the language itself).

You can try it out via the quick start: GitHub - Zokka-Dev/zokka-compiler: Fork of compiler for Elm, a functional language for reliable webapps.

Or if you’d prefer to use npm, you can directly run npx zokka.

Its main goal is to offer an easy way for the community to take advantage of work on patches to core libraries and compiler.

To do that it offers:

1. Custom package repositories (one is provided out of the box but you can set up your own if you would like)
2. Dependency overrides: for any direct or indirect dependency, you can override it with another API-compatible package.
3. Bug fixes to the Elm compiler itself (e.g. sanitization of strings when generating index.html files and fixing two bugs related to tail-call optimization)
4. No changes to the Elm language itself. Should be drop-in for an pre-existing project (let me know if this is not the case!)

If you have npm installed, you can go through a 60 second demonstration of one of Zokka’s bug fixes:

Open up the 0.19.1 Elm repl.

npx elm repl

and the paste in the following tailrecursive function (this is a variant of the factorial function)

tailrecursive value cont =
    case value of
        1 -> cont 1
        _ -> tailrecursive (value-1) (\result -> cont (result * value))

followed by

tailrecursive 2 identity

This will result in a stack overflow. This is extremely puzzling because we have a tail-recursive function that should take constant stack space!

Do the same thing with Zokka.

# Paste in the same code
npx zokka repl

You should see the correct answer of 2.

It plays nicely with other patches to core libraries. E.g. if you’d prefer to use Elm Janitor’s packages over the default one provided, you can do so! See GitHub - Zokka-Dev/zokka-compiler: Fork of compiler for Elm, a functional language for reliable webapps.

It also plays nicely with the standard Elm compiler itself. You can use the two freely on the same codebase, without any need to manually clear caches. If you see any bugs related to that please file a ticket!

There are some nice side benefits that come from Zokka’s approach, e.g. you can use an internal company repo for CI, indeed you can force Zokka to only use your internal company repo.

It currently is labeled alpha because the code is messier than I would like, error reporting is jankier than I would like. However, the compiler can handle real-world codebases. I use it on my own projects with no problem.

[konnik]

Wow, this is very cool. The possibility to use a private repository for packages is a killer feature in a corporate environment. I’ll definitely keep an eye on this project. Good work!

[Magnetize2072]

Very cool! A few questions

1. Is there anywhere listing the patches you have made to the original compiler? E.g somewhere one could see that you’ve brought in fixes X, Y, and Z.
2. Anything to share about the name Zokka?

[yosteden]

There’s a list of some of the bugs I’ve fixed here: zokka-compiler/FIXED_BUGS.md at master · Zokka-Dev/zokka-compiler · GitHub

I’m currently discussing with people whether there is a better way of doing this.

As for the name Zokka, there’s no semantic meaning. I was just looking for something that is fairly easy to pronounce for a large range of people, fairly unambiguous as to its pronunciation, has low SEO conflict, and is short and somewhat memorable. “z” sounds exist in most languages but are somewhat uncommon in the English-dominated sphere of technology terms, although certainly more common than among English terms unrelated to technology, so it helps somewhat with SEO.

“o” and “a” sounds are fairly common in most languages and natural choices for vowels. Finally the term “okka” is pretty unambiguous for most speakers as to what it should sound like. The double “k” helps remove some of the ambiguity of what the “o” should sound like for English speakers and in general “k” represents more or less the same sound across languages that use the Latin alphabet. If it was “Zoka,” there would be some ambiguity as to whether the “o” is pronounced like “o” in “token” or whether it’s pronounced like “o” in “sock.” Having the double “k” makes it clearer that it’s means to be the latter.

Also this is meant to not include the word “Elm” in it whatsoever (this was particularly important after some helpful feedback for earlier iterations of the project) to make sure that it is absolutely clear this is a community-driven project and doesn’t have official Elm endorsement.

[HuwCampbell]

Is there an option for local packages as well? Like if I have a set of packages under development on the machine?

[yosteden]

@HuwCampbell there is, but it’s slightly roundabout. Basically, you’ll need to zip up your project into a zipfile and then expose it over localhost before adding it to your custom-package-repository-config.json.

Concretely it would look like

zip -r mypackage.zip mypackage

# Note this and use this output in custom
sha1sum mypackage.zip

# Whatever you want to use as a local server
python -mhttp.server

and then add to $ELM_HOME/0.19.1/zokka/custom-package-repository-config.json (where $ELM_HOME is usually ~/.elm.

{
    "repositories": [...],
    "single-package-locations": [
        {
            "file-type": "zipfile",
            "package-name": "whatever-author/whatever-package",
            "version": "1.0.0",
            "url": "http://localhost:8000/mypackage.zip",
            "hash-type": "sha-1",
            "hash": "the_sha1_hash_from_the_previous_step"
        }
    ]
}

and then add to your elm.json the whatever-author/whaterver-package at version 1.0.0 dependency. The exact author, package, and version strings don’t matter in this case, just make sure that the repository configuration and your elm.json agree. Note that if you change your package during development, you should also increase your version number here. (Or you can simply delete the Elm package cache at $ELM_HOME/0.19.1/packages).

Also note that if you don’t need to change your package, you only need to do this once for as long as you don’t disturb $ELM_HOME. You don’t need to keep that web server running after the initial build.

You can also add the package path as an additional source-directories entry, but that won’t pick up on the transitive dependencies and other settings in the elm.json of your local package, whereas the above approach handles all that.

There is the very reasonable question of “Why can’t I just directly put in a file path instead of zipping up the package and serving that?” Zokka currently follows Elm in aggressively caching package resolution as much as possible, because this helps immensely with build times. This means that it downloads packages to a global cache once and tries to reuse that as much as possible.

Putting a file path usually indicates that the user wishes for the package manager to “watch” that path, so that it can pick up on any new changes there, but that’s quite difficult to do in the current caching setup, and in general is quite difficult to do at the project level, because you essentially want multi-project compilation (since presumably it would be nice if changes in the local library were compiled automatically to be used in the descendant executable). This gets very messy very fast and requires a lot of dedicated work to do well and even then still often has a lot of sharp edges. This is especially tricky to do while preserving Elm’s lightning-fast compile times. It’s still possible, but it’s quite hard and I’d likely only try to tackle it if there was an extremely large amount of interest.

[yosteden]

There’s also a question of “even if I have to zip it, why do I have to spin up an HTTP server and provide a SHA-1 hash?” The latter is because having a SHA-1 hash is a good way of reminding the user that the package is cached once and reused a lot, so that they aren’t surprised if they replace the zipfile and don’t find changes being picked up right away (again unless you change the version number or remove the package cache). The former is truly unnecessary and depending on how much demand there is, I may build out something that lets you use file:/// URLs directly.