@avh4 shares the origin, history and plans for the future of elm-format and a new, related tool!
11 Likes
Awesome episode. 
Here are some remarks from me 
29:39 : That statement is actually wrong, you can use whatever language to implement the backend of your vscode (or most other IDE) extensions
so we could just go and get the haskell parser, if that would solve the problem (it probably doesn’t, it will probably needs changing). at this point, I feel we’re approaching “parser bloat”, everything seems to parse on their own, nothing seems reusable.
- elm
- elm format
- elm-analyse
- elm-review
- elm-lsp
all parse on their own. I think the parser elm wants is different from the ones the others want, so that is probably fine.
the thing aaron seems to describe at 30:00 sounds to me like it might end up like https://microsoft.github.io/language-server-protocol/specifications/lsif/0.5.0/specification/ but in a non standardized way. I might be wrong.
If you anyone want’s to talk about this or work on this, feel free to reach out to me.
3 Likes
Absolutely! Loved it too!
I have some thoughts on this:
The haskell situation
This is the exact feeling I and many other had in the haskell ecosystem. The situation wasn’t so bad: There existed the 3 most popular parsers:
- The GHC (Pretty much only viable commercial haskell compiler) parser
- The
haskell-srcparser - The
haskell-src-extsparser
There is a lot of valid haskell source that GHC accepts, and it’s very complicated to parse. It depends on what language extensions are enabled.
haskell-src only covers haskell without language extensions, which pretty much doesn’t exist in any bigger code base.
haskell-src-exts tried to cover haskell with all language extensions but was very hard to maintain, leading to it being behind in features from time to time.
In the haskell world people have agreed to build upon the ghc parser and expose it as a library. They added features to it like keeping track of how tokens and AST elements related so they could parse code, change its AST and write it back into a file.
The Elm situation
In Elm, we now have many parsers. I think this is fine, for following reasons:
- Elm’s syntax is very simple, it’s easy to create new parsers
- Elm’s syntax changes infrequently (as opposed to maybe 4-6 months changes in haskell on average, although edge-cases, admitteldy)
- Different parsers have very different goals:
elm/compilertries to throw away unused information, be fast for non-incremental parsing, only needs to doc-comments, can’t parse incorrect code and tries to give the best syntax error messages possibleelm formatonly preserves certain newline positions and most comments, doesn’t need good error messages (elmhas got us covered there) and needs to write back to text.elm-lspneeds to parse incrementally (on every keystroke!) and even parse partially incorrect source code and doesn’t need to provide good error messages, but also needs to write back to text.
Conclusion
I don’t think having many parsers is bad. I still think a source of truth would be great! Why not create a public standard grammar that gets updated every release? This is something that Dhall does. Implementations can generate parsers with certain properties for their specific needs.
I bet there’s more to argue with this. For example: What about sharing ASTs? This could make sharing a source-of-truth for import semantics easier, for example. But I’m not sure what conclusions to take in that case.
4 Likes
I think I just want to (over) optimize it, as every parser running on it’s own just wastes cpu cycles/energy. And even if it’s “simple”, it needs to be maintained so more people to maintain something are always better. So it would be nice to share that work, somehow. For e.g. I think that your definition of elm format and elm-lsp sound compatible.
Even if I don’t think that not having good error messages in elm-lsp is an option in the long run. I think the intelliJ plugin is basically trying to move as much logic as possible to internals and not rely on outside binaries as much as possible. So moving more logic from X into elm-lsp is an option.
We might also get some of this for free from tree sitter for e.g. https://github.com/tree-sitter/tree-sitter/issues/255
1 Like
Helpful parser error messages · Issue #255 · tree-sitter/tree-sitter · GitHub sounds great but would only be the bare minimum in terms of error message quality one would like (compared to the elm compiler).
Yeah, you’re right. I think there is some potential to share parsers.
I also did some more digging and found out, that e.g. elm-analyse, elm-review and elm-lens all use the stil4m/elm-syntax parser (the one written in elm, which had a talk about it). So in terms of elm parsers the list is basically:
- elm/compiler
- elm-format (derived from an old version of elm/compiler, so fairly similar)
- stil4m/elm-syntax
- elm-lsp, the tree-sitter
I don’t understand what you’re saying with this, but I want to. The first sentence makes sense to me, but what do you mean by “X”?
1 Like
There is also the one in progress from elm-in-elm also written in Elm.
1 Like
Agreed, but that’s just what you would get for free, nobody keeps you from improving it more
At the moment elm-lsp relies on you to install elm-format, elm and elm-test in a compatible version and so that it can find it somewhere. First of all, that quiet a bit of setup and assumptions. And if these tools aren’t up to what we expect from them, moving functions from them into elm-lsp/intelliJ is likely going to happen.
One example for that would be the type inference. The removal of that api caused it to be reimplemented in intelliJ.
The only way to show “elm errors” while you edit, without saving to disk is having your own parser and creating those errors on your own.
1 Like
This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.