I’ve used elm/parser to build a substantial parser of a markdown format, and the result is really great.
However, the use-case involves routinely parsing several hundred page documents, and while the parsing takes only a few seconds, it still locks up the UI for several seconds.
The obvious solution is to move the parser to a web worker, and parse asynchronously. And this definitely improves the ui lockup situation, but deciding the rather substantial Json that comes back from the webworker takes a still noticeable amount of time, during which the interface is locked up.
I wonder if anyone has a way to pass the data from the worker to the main thread without an intermediate json stage, since their both elm, or if there’s a way to make the original parser (in elm) operate concurrently using Process.
Instead of moving the work around (to a web worker, to a process, etc) can you defer doing the work until it’s needed? You know the application best, but it doesn’t strike me that people will be looking at several hundred pages at once. Could you parse a page or two, or ten, and wait to parse more until they’re close to the current viewport?
Is it possible to break up the parser by having it do some parsing, then return a continuation to do more? Then you can interleave parsing work with a Cmd to run the continuation, which allows the UI to process queued Cmds?
Note that SearchResult allows a continuation where there is still more searching to do:
type SearchResult state
| Goal state (() -> SearchResult state)
| Ongoing state (() -> SearchResult state)
I experimented with using nextN with batches ranging in size from 10 to 10,000. With small batches the UI remained responsive, but the search did not run very fast. With large batches I could get better performance out of the search, but the UI was very unresponsive. I concluded that if you really need a burst of 100% CPU work that will last longer that a few milliseconds - you are going to need to look into web workers.
As far as I know, you cannot just pass anything between the main UI thread and a webworker when both are built with Elm - you do it through ports and need to encode via JSON.
A web-worker is not treating a function as a first-class function in the sense of FP. That is, to be able to pass around functions like any other type, FP captures functions as continuations - the code + the context (stack) in which the code is run. Web-workers run in separate contexts not shared with the caller. So web-workers are more like separate processes running on the same OS, rather than separate threads within a multi-threaded program.
If you can’t pass functions, you can’t pass any Elm type. Therefore it seems unlikely that we can ever do this - does Web Assembly support threads?
If you can find a good way to divide things into chunks then Process.sleep 0 should get you what you’re looking for. This should allow other events that the browser has backlogged during parsing to be processed before moving on to the next chunk.