What is currently impossible to build with Elm?

[max]

Hey everyone!

I was wondering if you ever ran into a situation where a certain problem was impossible to solve with Elm, even when using flags, ports or web components? So by impossible I mean there is no JavaScript solution and excluding altering Elm’s output or the compiler itself.

Whenever Elm is criticised, the critique is usually not because something wasn’t possible with Elm in general, but because something wasn’t possible in Elm only. (especially if the person has tried to contribute their solution)

At my current company we are aware of the things that Elm currently can do natively and happily choose it when possible or fall back to JavaScript via its FFI. Nevertheless I was wondering if there are problems that cannot be solved by the JavaScript interop that Elm provides.

I’ve heard of some issues with DOM events that need to be called synchronously, which apparently was solved in 0.19 (any links to the issue would be appreciated) and maybe rehydration of DOM listeners after server-side rendering is not possible without altering the compiled Elm code itself?

I’m mainly interested to learn about the tradeoffs that Elm’s FFI has, because despite requiring more boilerplate I see way more advantages than disadvantages. (which makes me wonder why other languages don’t choose the same model)

Thanks!

[mattpiz]

I’d say it’s not always a question of what can’t be done in Elm, but rather what can’t be “reasonably” usable if done with Elm. For example, two years ago, I was working on image annotation for creation of learning datasets for computer vision. Visual feedback is a powerful tool when you give instructions to people and one such instruction was to draw a bounding box over some object of interest in images. But if the user performs the task wrongly, I wanted to show them exactly why their interaction was wrong so that they do better on the rest of the dataset. Such visualization would look like follows

However, in order to compute that “mask” I had to perform topological operations (intersection, unions, etc.) of a bounding box and a mask encoded in a RLE (run length encoded) format. Those algorithms are done on matrix data, of similar size than the image size in pixels. I had two choices and tried both. You can either

1. do the entire algorithm in Elm, relying on Elm data structures
2. pass through ports back and forth

Unfortunately both options were not usable, due to the time it takes. In the case of doing all in Elm (1) the persistent data structures are not adapted to math and pixel-like manipulation due to there persistent nature. And passing such big values through ports as list or Array involved lots of conversions on “big” data and was taking too much time to be usable in that context. So unfortunately I decided to remove the visualization to be able to keep Elm.

That’s just one such example but I’m sure there are many more at the frontier of what Elm is good at currently. I just learned to be patient and keep contributing to Elm where I can to slowly expand that frontier. It’s just frustrating that it takes “years” sometimes but I’m ok with this.

[gampleman]

• I would second this. Specifically algorithms which are tricky to implement in a pure language but which are at the “core” of some UI tend to spoil the broth. At that point you may need to move a significant portion of your application to JavaScript.

• Another limitation I faced was needing to render to texture when doing WebGL. This is simply not supported in elm-webgl. If your whole application is based around rendering something in WebGL, there is no point using Elm for it.

• Some applications (quite rarely) actually need some more advanced web platform features for their architecture. For example if you wanted to build a modern mapping library, you would want to have shared array buffers for sharing between web workers and GPU processes. This is not suited to Elm at all.

[alexkorban]

I’ve seen an example from @toastal that didn’t seem to have a good solution: “In today’s episode of [Elm] limits: I have to rip out a bunch of code to calculate MD5s on files. Any MD5 library using Bytes will block the main thread. I’d send the File or Bytes out a port, but they’re not port-compatible. Now reimplementing all of File picking outside Elm.”

[antew]

Do you know if it was with a custom element? I had one I made a while back for encryption, and I adopted it for md5 hashes, https://ellie-app.com/8NZPSgYmTZGa1.

I guess this approach would be considered a solution outside of Elm. it is very similar to how the kernel code handles it in elm/file, but it isn’t Elm itself.

I find myself writing custom elements all of the time now for things that are unwieldy in Elm or require ports, the downside being that runtime exceptions are possible again.

class CustomFile extends HTMLElement {
  constructor() {
    super();
    this.processFile = this.processFile.bind(this);
  }

  processFile(evt) {
    const file = evt.target.files[0];
    const filename = file.name;
    const reader = new FileReader();

    reader.onload = (e) => {
      const data = e.target.result;
      const hash = md5(data);

      const evt = new CustomEvent("file-selected", {
        detail: {
          file: file,
          hash: hash
        }
      });
      this.dispatchEvent(evt);
    }

    reader.readAsArrayBuffer(file);
  }

  connectedCallback() {
    const input = document.createElement('input');
    input.type = "file";
    input.addEventListener('change', this.processFile)
    input.innerHtml = "Open file";
    this.appendChild(input);
  }
}

[alexkorban]

Not sure. It’s certainly a solution, but with custom elements, I think the question is where to draw the line. At one extreme, the whole app can be stuffed into a custom element

What I take from this example is that if you don’t know upfront that you have to implement that bit of functionality with custom elements or JS (or requirements change later), then you might end up reimplementing stuff, and also that there are types of data which cannot be sent to JS via ports.

[pdamoc]

Custom elements are a reasonable solution and they can be used for a lot of the problems that might arise.

Most users will seldom run into limitations and out of those who run into a limitation very few will run into a problem that cannot be solved by ports or custom elements.

And in the very unlikely case that a user runs into something that cannot be solved by ports or by custom elements they still have the options that involve patching either the output or the core libraries. I have not yet run into such a problem but if I will do, I know how to approach it.

[harrysarson]

It is impossible to build a program in elm that solves the halting problem https://en.m.wikipedia.org/wiki/Halting_problem.

Cant blame that one on a lack of FFI though.

[max]

Haha yeah I guess my definition of impossible can be refined further. I don’t expect it to solve unsolved computer science problems.

[max]

Thanks a lot for all the replies so far!

I guess the examples could be summarised with problems that require the browser’s low level API for performance reasons, which cannot be done in a performant way with decoding/encoding or in Elm. Maybe these can be solved in the future when Elm supports WASM as a compilation target.

[hanifhefaz]

I think this one also goes into the list.

[toastal]

I actually need to hold onto this file for later consumption (calculate MD5, check if exists, then upload the file to multiple source, and report the upload various responses) which would mean this custom element might need to live up at the root and just exists as a very hacky proxy to circumvent limitations. Sure it could work, but being able to at least send a File or Bytes through a port would seem like a more obvious solution.

I’m instead writing this whole bit in PureScript’s Aff and subscribing to Elm ports as events and keeping the file’s ‘state’ outside Elm but in ethereal Fibers waiting for cancel subscriptions that can run in parallel. Alongside managing the various progress events, the most cumbersome bit though becomes needing to Json.Encode.object, Foreign read/write, Json.Decode.decodeValue my ADTs between the boundaries. If this path gets hairier, I would pull in something like purescript-kancho to help with this part.

[antew]

I may be missing something, but why would the custom element need to live at the root? Once the file has been selected the File and hash are sent to Elm and it could hold on to them.

I agree there is an impedance mismatch on how files are handled through ports, there are ways to send files in through ports easily, but sending it back out is a pain, it would be nice to be able to send a File out through a port directly.

For sending in through ports it can be done like

      processFile(evt) {
        const file = evt.target.files[0];
        const filename = file.name;
        const reader = new FileReader();

        reader.onload = (e) => {
          const data = e.target.result;
          const hash = md5(data);

          app.ports.gotFile.send({
            file: file,
            hash: hash
          });
        }
        
        reader.readAsArrayBuffer(file);   
      }

and on the Elm side

port gotFile : (Encode.Value -> msg) -> Sub msg

subscriptions : Maybe (Result Decode.Error ( File, String )) -> Sub Msg
subscriptions _ =
    gotFile (Decode.decodeValue decodeFile >> FileSentThroughPort)

decodeFile : Decoder ( File, String )
decodeFile =
    Decode.map2 Tuple.pair
        (Decode.at [ "file" ] File.decoder)
        (Decode.at [ "hash" ] Decode.string)

I made an Ellie with it, it still uses the custom element for the file input so that it can get the MD5 easily, but the file is sent back through a port. https://ellie-app.com/8Q6WF5NvSR7a1

For sending out through ports it is another matter, depending on requirements window.URL.createObjectURL is an option to turn the file into a String we can pass around more easily and send out through ports. We use it in our app for video upload, when you select a video the File along with the object URL (which looks like blob:https://your-doman.com/some-identifier) are sent in to Elm, that blob URL can then be as the src in a <video> element.

You can pass the blobUrl, filename, and mime out another port and reconstruct the file that way, but it is really fetching the file again and creating a new one, and you need to handle calling revokeObjectURL at some point too so that memory is free’d up.

I made an Ellie that lets you select a file, sends the File and the MD5 in through a port, sends the file back out through the port as a blobUrl, and then back in through another port as a new File
https://ellie-app.com/8Q8m52Xr7SRa1

The meat of the code is

    // Sending the file in 
    app.ports.gotFile.send({
      file: file,
      blobUrl: createObjectURL(file)
    });

    // Receiving the file and sending it in through another port
    app.ports.sendFileOut.subscribe(({blobUrl, filename, mime}) => {
      fetch(blobUrl)
        .then(res => res.blob())
        .then(blob => new File([blob], filename, { type: mime }))
        .then(app.ports.gotFileAgain.send)
    });

[dmy]

Note that the file can make a round trip “as is” to and from a port as long as you keep it as a Value.

I updated your second example to show it:
https://ellie-app.com/8Qb6JRwqqZpa1

[antew]

That’s cheating! Just kidding, that is a good point, I guess you could treat it as a (File, Encode.Value) to get the best of both worlds. The objectURL string is mainly useful as a src in image or video elements.

[dmy]

I don’t think you can do anything more with a File than with a file as a Value, because you can decode the Value into a File anytime, and you cannot modify a File anyway.

For example, you can implement your first example that selects a file, compute the md5 and get the file back into Elm without a custom element, selecting the file from Elm, and just computing the md5 in javascript:

https://ellie-app.com/8QdkbTyvnzBa1

The only limitation is that you cannot use File.Select.file, an input [ type_ "file" ] must be used instead.

@antew

I guess you could treat it as a (File, Encode.Value) to get the best of both worlds.

Yeah, it could be nice to have a package that encapsulates them in an opaque type, and exposes the elm/file API, with an additional encode function. File.Select would be replaced by File.Input.

Would this be useful? Should this be published? Maybe there was a good reason an encode function was not included in elm/file

PS: sorry to derail the thread, let’s discuss this elsewhere.

[toastal]

Both examples have UI that hangs for me. I’m dealing with video files that can be in gigabytes. The MD5 calculation and the createObjectURL has to be done in a Web Worker at this size. Because of the file size, it’s not very reasonable to be turning into and out of a string unless absolutely necessary. I have no hangs in my set up.

I wanted to use Elm crypto libraries to do the MD5 calculation. I guess it could be piggybacked into a Web Worker using Platform.worker, pushed a File blob out a port, to the JS, to the Web Worker, to the Elm worker, and back out again. But now you’d have to hold onto that file somehow in the Elm side waiting for the async response (and there are no WeakMaps to use files as keys), or do this expensive process of sending the whole file in and out of the port. In PureScript, I was able to wrap the Web Worker in an Aff sending in the actual File, and then use that worker as if it was any regular async computation without a bunch of unnecessary marshaling steps.

Why would the element need to live at the root? Because in a SPA you replace the page DOM and you don’t want this action/element to be last just because you switched pages.

[dmy]

@toastal Passing the file as a Value does not require to turn the file into a string or to change its format whatsoever, and you can use a web worker inside the port.

I cannot put it in an Ellie because web workers are not supported there (the iframe does not have the allow-scripts and allow-same-origin attributes), but you can test the simple following example that allows to compute asynchronously and simultaneously several file MD5s in chunks of 2MB from a webworker through a port:

https://rlefevre.github.io/elm-chunked-md5-webworker-port/

You can open several GB files and look at the javascript console to see the progress.

I tested it with a 27 GB file and several smaller other ones simultaneously.

Notes

• The JavaScript File is selected from Elm.
• This file reference is then directly sent through a port and is not hold into the model in Elm side until it is received with its MD5.
• No custom element is used.
• The files are actually read only once in chunks of 2MB to compute their MD5, the application memory footprint in Firefox was around 3 MB at anytime during my test.
• Of course I would prefer to do it in pure Elm, but web workers require ports anyway to communicate and reading a file in chunks is not possible yet as far as I know.
• The whole program is 116 LOC (61 Elm + 22 HTML + 33 JavaScript counted by cloc).

Sources

• Main.elm
• index.html
• worker.js

[mgree]

Many folks have referenced UI hangs as a showstopper for Elm.

I’ve been working on an Elm library for suspendable computation (GitHub repo). I haven’t released it yet, but it’s nearly usable.

It’s not all roses, though: to use my library, you have to change how your program is structured. You can either use a Fueled type that amounts to a step-counting monad—which means your code is littered with |> andThen \v -> ... or writing your long-running program as a rewrite system, i.e., a function step : Config -> Either Config Result.