Where is Elm going; where is Evan going

Request Feedback
1

INTRODUCTION

Although I have been thinking about the subject of this thread for some years, this thread is triggered by a presentation that Evan gave a few months ago, where along with analyzing the economics of funding new computer languages/projects, he reveals that he has been working on a language that has a C back-end that generates PostgreSQL tables and asks for input on what to do with this work.

SURMISED EVAN’s PROJECT

Given his background as Elm BDFL and functional language evangelist, we can surmise that the language resembles Elm and may be an adaptation of the Elm compiler written in Haskell and that, given Elm’s preparation for a reference counted memory management scheme in not allowing non-global cyclic references, it may use reference counting for memory management (or not, it doesn’t really matter), we kind of get an idea of what his demo project might be like. In his talk, he revealed that this project wouldn’t have some of the restrictions that Elm has as “decisions made about the Elm language are unique to what he sees as Elm requirements and aren’t necessarily as he would like to see general features” (paraphrased) and revealing that for a new language/project that the answer to general requests for features is generally “yes”.

WHERE DOES EVAN’S PROJECT GO?

I have tried to open a discussion which him by email as invited in the talk but have not succeeded in this, which discussion might be triggered by this post. In his talk about the economics of development, he generally seems to favor “hosting” by companies using the product, which is the general principle of Elm funding and works for a niche project as is Elm and I suppose would work for another niche project as in this new one if there are companies/a company who see a use for such a project. However, a PostgreSQL table generator is not the direction I would like to see a new Elm-like/Elm-based project go and in just Yet Another Niche Language (YANL) just as Elm is YANL whose only use is generating JavaScript-based web pages: my ideas are that to be more generally successful, a new Elm-like language needs to be more general purpose and embrace general use for all of the niches one might envision just as successful languages such as Python, Rust, etc., encompass a huge range of uses through packages with a minimum of custom support by the compiler.

MY IDEAS

Accordingly, I see that an Elm-like language could fill a more general set of needs, as follows:

  1. It would be backward compatible syntactically with current version 0.19.1 Elm so that all/most current Elm packages work with it other than the few “native” “elm” and “elm-explorations” ones that would be re-written to use C as its “native” back-end language.
  2. Since it generates C code, it could be used with Emscripten to generate JavaScript and web pages so be immediately useful to all current Elm users, but thus adding the ability to be compiled to Web Assembly for a whole new range of applications. Alternatively, it could have several options in selection of back-end to generate JavaScript directly but other languages as well as does Fable, an F# transpiler that now is moving to support output of Typescript, Python, Dart, PHP, and Rust, as well as JavaScript.
  3. Versions of the equivalent to the VirtualDOM could be written for all major platforms to run under WebKit and thus support GUI’s for at least Windows, MacOS, Linux, Android, and IOS (as per the Rust Tauri project) and thus serve a role as does Google’s Flutter, and could also support CLI’s for all of the platforms that support them.
  4. Generating PostgreSQL tables would just be yet another package added to the new language’s ecosystem, and thus the new language could also fill that niche requirement.
  5. Some missing features such as full FFI could be added without breaking Elm syntax compatibility; as well, some desired features, although not strictly required, such as Higher Kinded Types (HKT’s) and some sort of type classes could be considered, which then would make it fairly easy to support something like ST modifiable linear arrays easily (although this last does not strictly require HKT’s and type classes).
  6. The first version would likely not be very sophisticated as to optimizations (as neither is Elm), but further refinements could include optimizations eliminating redundant reference counting by using data flow graphing (as do the languages Nim and Lobster), which can greatly speed up reference counting; as well, speed bottlenecks can be eliminated by using the common functional language programming techniques of function inlining, lambda “lifting”, phantom argument elimination, etc. to produce C-speed code.
  7. Something could be done about Elm’s incomplete Process specification (currently doesn’t permit inter-process messages other than as used by the compiler when using effect managers) which hasn’t been a big problem for use generating JavaScript which isn’t multi-threading and sees limited benefit from a concurrency model; being more general purpose, the new Elm-like language needs both concurrency and parallel execution models.
  8. The result would be a pure functional language which is much easier to learn than Haskell (simpler stripped-down syntax and strict-by-default) while not having all the baggage of F# having to support DotNet with ancestry back to OCaml with a deterministic memory management system which could attain the speed of raw C code with the general safety of Rust without an obscure new syntax (just general functional programming as in ML languages).

WHERE DO WE GO?

Evan would likely be the best BDFL for such a project due to his Elm background, but if he doesn’t see it the way outlined above, then it could be forked away from Elm while still attempting to manage backward compatibility. If this new project adopted a new contributor-friendly policy (unlike Elm), I would happily be a contributor to the project. I am currently doing some work on an alternate to the self-hosting compiler library “Elm-in-Elm” project (which project I don’t see being completed anytime soon, given that it wants to do everything from scratch) and have translated large parts of the Elm compiler written in Haskell to Elm, with the parser completed, and I’m pretty much on top of the canonicalization, checking, optimization, and code generation phases, with not so much done on the type inference phase. This last phase likely requires the most work as it seems most non-trivial Elm compiler issues are related to type inference, I think possibly related to current Elm not unifying type variables from global scopes across nested local scopes.

Although a self hosting Elm-like compiler would be something nice to have, it isn’t absolutely necessary and just modifying the current Elm compiler written in Haskell starting with the back-end to generate C would also have the intended result.

WHO’S WITH ME?

Given that Elm hasn’t seen any releases for over four years now, even as to bug fixes, and loathe to see the incredible Elm functional syntax die or be relegated to only use in the current or expanded niche purposes, do any of you see things as outlined above or have further ideas to contribute. Even more specifically, if there were such an open source project, would any of you see yourselves as contributors or available to head the project?

1 Like
2

Elm is not general, and makes ffi hard on purpose to limit wrapping js libraries and inventivice pure elm solutions. things that makes it great. I think several of the points on the list is non elm goals.
But more constructive: have you checked out roc-lang and gren-lang? Both based on elm and might align closer to what you describe.

3 Likes
3

@Atlewee,

Yes, I understand and and accept the FFI limitations placed on Elm for it’s use; what I am proposing is a more general purpose Elm-like language whose back-end is C and therefore would not need those limitations.

Again, that is the point of the proposed Elm-like language: a general purpose language that is syntax compatible with Elm and can do what Elm does, but more…

Yes, I’m familiar with both of them but Gren is only a slight modification to Elm still producing JavaScript.

Roc is more interesting and has many of my ideas of a general purpose language, BUT IS A WHOLE NEW LANGUAGE, with all the problems that new languages have. It is also written in Zig which seems to break functional languages principles in writing a functional language using an imperative one. Why invent a new functional language when there already exists an almost perfect functional language syntax in Elm?

4

I think it’s worth noting that he isn’t working on a separate language but on Elm itself. The language that has a C backend and generates PostgreSQL tables is Elm.

I think this changes a lot of your assumptions and such.

3 Likes
5

@wolfadex,

How does being Elm actually change my assumptions? I said that Evan’s language is most likely a modification of the Elm compiler and thus just Elm with a different back-end and core packages and all of my ideas are based on an Elm-like language.

Where my ideas may differ from what Evan’s seem to be currently is I would like to see a unifying general-purpose Elm-like language instead of what appears to be two separate efforts, one current version that generates web pages and another that generates C for PostgreSQL tables.

If Evan has already gone through the exercise of generating C from Elm, it isn’t all that much a bigger step to make it a general purpose language as I outlined. I suggest that in order to succeed as a general purpose language, it needs to have a more contributor friendly development, which being a more sponsored development rather than mostly hosted, may make it less subject to being “Jeff’ed” (as per Evan’s talk from the link in the OP).

6

A new version of elm targeting what you talk about will also be a new language just like roc and gren.
From what I understand they both started out with 100% elm syntax/mental model and changed parts to solve those language goals better. The same thing would and should also happen if you change the goals of elm.

Going via C for everything might not be the best solution, designing a language is a constant struggle to balance compromises. You can potentially get better runtime performance (if targeting webassembly), and code once/run everywhere, but at the cost of compile times and bloated bundles and build system.
A good compromise for servers/applications/games maybe, but if you need to serve stuff on the web as fast and minimal as possible, directly targeting js is probably the optimal solution today.

It might be a really nice solution you describe, you should build it.
( But be aware that Roc is more or less already doing exactly what you describe. Starting of with elm syntax and make it a ultra-performant general easy to learn functional language )

( Roc uses low level imperative language (rust+zig) to do stuff like in place mutation under the hood for performance reasons if it is safe to do so (only one refcount).
The cpu / assembly / bytecode is imperative with manual memory management, so to get the most out of performance i guess rust and zig is a great choice for roc )

2 Likes
7

I think maybe I’m not understanding your post. I think maybe on a second read you’re wanting Evan to abandon Elm to build a new language that’s like Elm but with features you want? But I’m not sure. I think there’s a fair amount of misunderstanding about his current work that have influenced this post, but it’s hard to say.

@Atlewee is right that Roc is basically the language you’re describing. I’d also argue that you’re describing Haskell, and maybe even PureScript.

1 Like
8

I think I’m also confused because the things you talk about building, desktop apps and CLI tools, are possible to build with Elm today

9

“It’s been a while since we’ve seen GordonBGood — their last post was 4 years ago.”

Hi Gordon, welcome back. I always enjoyed your posts many years ago, but I also got the impression you were put off by some of the design decisions and direction that Elm took.

I am interested to hear your thoughts on what this hypothetical Elm-like language would do with regards to parallel processing? On the browser it is single threaded so this is not an issue. A more general purpose language would likely want to take advantage of multiple threads and CPU cores. Do you have a model in mind that would mesh well with strictly typed pure FP?

I have been thinking a bit recently around things related to your points 2, 3 - WASM and virtual doms. Hopefully I will have something to report on this soon.

2 Likes
10

Welcome back Gordon,
I think most of your ideas resembles a fast, friendly and functional language such as roc-lang. In case of a full-stack platform, there’s lamdera with a proprietary model. There’s also elm studio which is still in development and I’m not sure what kind of development model it follows.

11

Little bit off topic… could you give me some hints about that? How can I build CLI and desktop apps with Elm?

12
2 Likes
13

@Atlewee,

Well, I’m pretty far into the specification of what such an Elm-like language according to my ideas would look like and so far haven’t run into anything that would break backward compatibility syntactically. Granted, the core “elm” and “elm-explorations” packages change but if only new API is added, it is still backwards compatible. It has been said in this thread that Evan’s new project is “Elm” because it uses a compiler based on the Elm compiler, but with a back-end of C it isn’t exactly “Elm” even if a compiler switch is used to switch to different back-ends as there must be package(s) to support the PostgreSQL table output.

Yes, there may be slight syntax add-ons to support “monadic mutation of linear arrays” and Haskell’s “do notation sugar” as Roc has, but those are optional and don’t have to be used unless these features are used.

Using C/Emscripten as an intermediate target costs the time of running another step in compilation but in practice is pretty fast, depending on the C compiler used (Tiny C, etc.) but gains from all the C optimization work done for compilers such as GCC and Clang over the past 30 years. The Fable language generates almost as fast JavaScript code from functional F# code as JavaScript written by hand, but I don’t think current Elm’s JS output is as good. Of course, that could change with further optimization development, which is why I mentioned that there might be a compiler switch to choose back ends. I just see the C route, which it seems Evan may already have in his latest project, would be a faster development for fast code that could be used from a general purpose language.

I have considered “building it myself” but at 70 years old don’t know that I have the energy and drive to carry it through; as well, if Evan has already done much of the “heavy lifting” in his latest project and can be convinced to expand his goals of Elm to be a general purpose language with a more contributor-friendly ecosystem, I would rather be a contributor to something that already exists. If this were to happen, one of my first major projects might be to make Elm self-hosting.

I have looked at Roc, but consider that it has diverged from Elm too early, although I am following it and may change my mind…

14

@wolfadex,

No, I don’t want Evan to abandon Elm or his new project that outputs PostgreSQL tables in C; I am suggesting to him and those who may have direct contact with him that the vision of Elm currently being limited to generating JS web pages, a niche use, or PostgreSQL tables, another niche use, might be expanded to become a general purpose language, as once the language generates C (for PostgreSQL tables), much of the work has already been done.

As a general purpose language and with a different development model, Elm might become more main stream.

Roc is too new to know what it might become, and in my mind has diverged from Elm too soon, although it does have the functional language aims I admire.

My favourite languages are Haskell, F#/Fable, and Elm, with my favourite being Elm if it were not limited to generating JavaScript.

I love Haskell, but it can never be my ultimate language as in too difficult to learn, non-strict by default, Garbage-Collected memory management, etc.; Purescript has all of the disadvantages of Haskell, gaining strict execution but losing as to less active development and an awkward package structure, while only generating inefficient JavaScript.

F#/Fable are nice but not “pure” functional and have too much DotNet and OCaml OOP baggage (although very capable for all of that).

Elm or an Elm-like language as a general purpose language has it all in being easy to learn, strict by default, “pure”, could be very fast outputting C, etc., although Roc also has this potential.

2 Likes
15

@wolfadex,

Yes, possible, but not encouraged by Elm main stream developers such as Evan.

For simple node-based CLI, one can just pipe console I/O through ports.

There may be more sophisticated ways to do this through packages that I haven’t kept up on.

I am talking about official core package and compiler supported ways of achieving these; if there are developments in packages along these lines, please instruct me.

16

@nov8d,

Thanks for the welcome back.

Yes, I have considered Roc, but think it may have diverged from Elm too early but should do more follow-up of its current state…

17

I would bet a lot of money that Evan understands this. Personally, I think he may move in that direction in the future, and it’s unclear how much is already included in the current phase of the project.

Also, I don’t agree with all of the features you assert should be part of a general-purpose language. Specifically, I strongly agree with Elm’s decision not to allow FFI (at least in packages). From a security perspective, it is very nice to have a language-level guarantee (and not just a promise from package developers) that packages can’t do anything underhanded (e.g. data exfiltration). To this end, I would even prefer if packages were disallowed from creating Cmds. For example, if application developers needed to pass Http.request to the package, they could pass a version that ensures that the package only sends data to the domain it says it is sending data to.

1 Like
18

@rupert

Hi Rupert,

I remember you in our discussions as well.

Indeed, I did have some intense discussions about some Elm language changes that I felt very strongly about, but I may have mellowed with age and gained some modicum of wisdom with experience ;-).

Things I argued as follows:

  1. I never had strong feelings about Elm’s lack of JS FFI and have come to Evan’s point of view.

  2. I argued that not having cyclic references could put a serious cramp in the use of certain programming algorithms, but have come to see that many/most/all? of those algorithms have more efficient alternatives that don’t require cyclic references.

  3. One point that I argued and about which I still feel strongly was the removal of the Lazy module that allowed the memorization of lazily computed values, with the reason given that it wasn’t performant (true in any language but especially as implemented in JavaScript); however, documentation explaining its limitations would have sufficed while leaving it available where it is an elegant solution - for instance, Haskell’s strings made of lazy lists of Char makes little sense generally, but lazy lists of arrays of Char might be the most elegant solution in some circumstances, which choice has been removed from us.

In the past intervening time, I haven’t abandoned Elm as it remains one of my favourite programming languages in spite of its limitations: I spent about a year contributing to Professor Christopher Anand of McMaster University’s grapicsvg as to writing testing modules and have contributed to many of the more complex RosettaCode examples such as the Y-Combinator, Church Numerals, Hamming Number Sequence, Sieve of Eratosthenes, and the Legendre Prime Counting Function.

I spent some time on this, but didn’t complete the testing: my feeling of what would work best for this was something like Haskell’s IOFork’s and channels for message passing adapted to Evan’s never completed Process stub where he proposed adding capabilities of inter-Process message passing. While this would never be as fast as “green” coroutines, it would be of reasonable speed at hundreds of thousands of “context switches” per second and only requires some not-too-major changes to the Task scheduler core module. I think this scheme could also support some sort of “async/await” on the Process Task’s but haven’t followed that up.

I am very interested in what you have to say on this. Please find a way to PM me or notify me in a new thread if this one has closed before you are ready.

19

My understanding of Evans presentation was that he’s working on a similar project as Lamdera and is unsure if he should release it (as it would directly compete with Lamdera or any similar project).

Elm is intended to be domain specific. And Roc is an attempt of making Elm general purpose. The changes made to Roc came from its need to be more general.

You won’t be able to “just make Elm compile to C” as that is either exactly what Node.js does or it would require making changes to Elm itself.

20

@Lucas Payr,

How do you know that the changes to Roc were necessary and that Elm would have to be changed to be more general? From the Roc documentation for Elm programmers it’s pretty clear that the Roc language designer(s) set out to create a “better” Elm that is still functional and never really intended that Roc be syntax compatible with Elm. Many or most of the differences exist to make the Roc syntax more concise or less ambiguous than that of Elm and some changes arise from the opinions of the Roc language designers, but none of them were a necessity to make the language general.

I have written a specification describing how Elm syntax could be used in a more general purpose language and the only changes identified would be those to support Higher Kinded Type’s (HKT’s, where the type variable syntax would have added syntax to be both less restrictive in definition but able to be constrained on use), something like type classes (Abilities in Roc), and for do notation sugar (which Roc also has), all of which can be used as a super-set of Elm’s current syntax and none of which are completely necessary but are conveniences. The core packages would likely be augmented, however, in order to support more general language features such as console I/O, linear arrays that can be modified by monadic function chains, multi-processing, etc. as added API without deletions or modifications to current API.

Someone else has said that Evan’s new project “is just Elm that outputs C” and if that is true, then new Elm must still be Elm syntax, although it would need added packages to do with PostgreSQL table generation.

1 Like