In this previous thread I asked for some suggestion on getting started with language extensions. I got some useful suggestions, which I am grateful for, thank you!
After going through those and also doing some further reading on my own, I revisited Microsoft’s overview on Roslyn. Through the example of .NET languages, it describes the overall scope of the problem of language extensions, refactoring, autocomplete and code generation tools in general, as well as their relation to the compiler logic.
It also helped me formulate a general idea of
what I can achieve by using a handicraft Elm parser (and some logic on top of that) – stuff like working with module/type/function declarations; generating code from parts of the AST etc.,
what I would need a larger part of the actual compiler logic for – like knowing the inferred types of expressions; understanding if the inferred and declared types do not match; evaluating expressions.
In an ideal world – where ideal is subjective to my perspective – I would prefer to use the actual compiler for both #1 and #2, but even in a less ideal world it would make absolutely no sense to try to do #2 without relying on the original compiler.
This leads me to my questions:
Did anyone try to interface with the compiler, or use the original compiler code to do anything I described in #1 or #2?
Is it even worth exploring this direction or I should just accept the constraints of #1 for now and pick a handicraft Elm parser (like this) or write my own?
I’m very interested in providing more type information to editor plugin authors. I’d like to do it through the compiler, and to make it really quick. (I think having slow editor plugins may be worse for Elm’s reputation than just not having them.)
So another path is to communicate clearly:
the APIs you are working with as an editor plugin author (i.e. what does Sublime Text offer?)
the APIs you think you’d like from a command line tool or server (i.e. what info do you need?)
Before everyone just says “Microsoft Language Server Protocol” again and again, I know about that. I’m not certain that a protocol designed for OO languages will be ideal for us. Please proceed by describing the specific things you (a language plugin author) would like to do with details.
For example, “My approach to rename refactoring will be to have an AST with X and Y characteristics. I need it to change like Z as the user types. I want to ask about W at various times. That will allow me to V. I need things to work in time T because I’ll need to lock P% of the project files to make those changes.”
That sort of thing. I need to really understand the domain to give good interfaces.
TLDR: I will make a plugin skeleton (for VSCode) and provide the details for some use-cases.
In the long version let me try to give you some context of where I am coming from.
Originally I had this idea of creating something for Elm, that’s similar to Scratch’s script editor. But instead of all those rather uniform blocks, let package authors define the blocks (or visual representations) for their functions. The point of doing this – other than just hopefully being fun – would be to convey domain knowledge in a more graphic and perhaps more interactive way.
For example:
In place of this UX:
You type Html.beginnerP
The editor shows a popup with the type annotation and the comment for Html.beginnerProgram
You accept the suggestion and fill in the record with a model, a view and an update
it would be something like this:
You find the pictogram for Html.beginnerProgram from a palette and drag’n’drop to some workspace (or do the same with the keyboard).
A block depicting the Elm Architecture appears, showing you the usual app lifecycle and perhaps the imitation of a browser window. There are blanks in it to fill with a model, an update and a view.
As you fill in the blanks it shows you if the types match up or not. When you filled in all the blanks your initial view would appear in the imitated browser window. Maybe you could send fabricated messages to your update and see how they affect your model (and through that the view).
The point is that this thing would aim for providing a more direct experience of the domain that the programmer is working on, regardless of what that domain is. And it would give package authors the tools and rely on them (since they are ideally experts of their package’s domain) to define those experiences.
Now, we (me and some other fool friends of mine) don’t know how to do this nicely. So we have to learn.
So I wanted to learn about the engineering challenges. And from an engineering perspective this whole idea sounds like a very rich editor plugin for Elm. So I thought it would be a good learning exercise to make a simpler, more traditional editor plugin first.
To do that I wanted to understand the domain of traditional editor plugins first, that’s why I started the previous thread.
This is what I have learnt so far
I read through much of the “Microsoft Language Server Protocol” specification. I think it roughly describes how a language plugin can communicate with an editor. So it roughly answers question #1 on editor APIs, but I will dig deeper into the VSCode API. Also, because LSP is what it is, I would not even consider it as a candidate for providing compiler support. It can be a candidate for writing parts of the editor plugin.
I refreshed my knowledge on compilers and parts of the pipeline
I remembered that I found Microsoft’s Roslyn project interesting. So I went through the overview again. Their goal there was to expose the knowledge that builds up in the compiler pipeline at different stages, through APIs to support a range of refactoring and analytics tools (including those built into Visual Studio). It is not a standard or a specification, and the final APIs are different even between VB and C#, but I believe that some of the ideas can be valuable. For example: their ASTs hold all the source information in full fidelity and errors are also represented, which makes their syntax trees completely round-tripable to the original text – even if it was a malformed text. I found those attributes particularly useful for editor plugins.
I will continue from here, probably make a plugin skeleton (for VSCode) and provide the details you asked for.