Without trying to step on anyone’s feet, I think the main issue preventing this from happening—or any other project rethinking Elm’s targets—is the fog around Elm’s long term economics.
As the talk highlighted, it is very hard to make independent languages economically sustainable. This is an issue Evan has had through the years (e.g. His Let’s be Mainstream talk), and tbh until this is ascertained everything else seems extra to me.
Yes, I would love to see an Elm that generally compiles to C. With the right incentive systems, the community could come up with very innovative ideas.
But for that to happen in the best terms, I think we have to show goodwill as a community, and appreciate Elm for what it is for now. This seems to be Evan’s life project (or at least, very long term one). And it looks like where Evan is going is where the project can most sustainably grow in his opinion
Yes, that seems to be Evan’s main concern at the moment.
Very well put. I think my concern here is that Evan has put considerable effort into this PostgreSQL table project that only moves Elm forward if that works to somehow support Elm’s current client base, and the only way I can see that happening is that Elm becomes more general purpose to be able to support both current users and new uses such as PostgreSQL.
The purpose of this thread is to explore whether you, those current users, have the same concerns, or if any of you have any indication from Evan on what his long term future view of Elm might be, and your thoughts on your ideal future for Elm.
I mostly take that assumption from things Richard shared on Software Unscripted. In particular, when he talked about his ideas of platforms. (don’t have a episode number for that, sorry)
For me, it seemed like Evan explained quite well how working on a language is an unthankful job and how he does not know how to continue. Elm Studio would have been his solution — but it seems like that didn’t really work out. So in my eyes, Elm only has a future if Evan gets paid for his work.
Until then, we as a community can build an ecosystem and find ways to improve the language. Gren is such an attempt, and so is Elm-janitor and Zokka.
Thank you for the thoughtful post, Gordon. I would also love the tools you describe (beginner and programmer friendly, general purpose, pure functional language). I also found interesting your thoughts on the “how”: like the required language features and choice of backend.
However, some aspects of the proposition seem out of touch given the history of Elm development and Evan’s more recent communication
Firstly, I think it’s increasingly clear how draining the open source work and hype over Elm has been to Evan. He seems to have found a way to work that suits him better, away from the spotlight and investing in long-term projects that match his vision. In light of this, I don’t think it is productive to rush to coopt his unreleased work.
As I understand this vision, Evan is trying to build a comprehensive experience to build on the web that is easy for beginners and delightful for professional. I think he might be aiming for a more accessible version of Ur/Web (author details benefits for web dev in this podcast). A general purpose language might not be the goal at all!
Devs wanting to get more out of Elm have taken matter into their own hands. There has been experiments and full-fledged frameworks and products to extend the use cases for Elm. I am a happy user of Lamdera which, like elm-page, provide an experience far beyond vanilla Elm. There is also Gren, for a more community oriented development and more advanced language features. And Roc for easy pure FP outside of the web. What I am trying to convey is that the FP landscape is a bazaar and while we might be longing for the cathedral, it is just too soon in my opinion. Better look for the project that’s more aligned with this vision you describe.
I have been looking a bit at the Lean4 compiler, which like Elm does not permit cycles (and hence, lazy data structures), and can use reference counting instead of garbage collection. Here are some slides and a paper on the topic:
I note:
“Destructive updates when reference count = 1.
• It is a known optimization for big objects (e.g., arrays).
Array.set : Array a → Index → a → Array a
• We demonstrate it is also relevant for small objects.”
So the compiler has auto uniqueness detection, following from rc==1, and can therefore frequently optimise updates into destructive updates.
Not sure what I think about lazy going away, but I’ve never missed it. Then again, its also not a technique that I have ever used much, so perhaps I don’t miss it simply through ignorance. However, the uniqueness detection optimsation sounds pretty neat to me too.
For me, it seemed like Evan explained quite well how working on a language is an unthankful job and how he does not know how to continue.
Yes, I get the sense that Evan has sort of burned himself out with extreme productivity over the first four or five years of Elm’s development and almost nothing over the past four years and concern with the financing of ongoing development. Perhaps he attempted to control too much of the language development himself? While sympathizing with him, it is a pity that development has slowed for those of us who very much would like to see continued development, even to more extended use of the language. This thread is kind of an exploration of how we might support him…
While I respect your thoughts on different directions one might pursue for a more advanced version of Elm, my general idea in this thread is to explore whether it is possible that Evan’s vision of where Elm is going might in some way match the direction that I would like to see it go. It seems that Evan may have a private version of Elm that, in generating C code, may accomplish a large part of what I think could turn it into a more general purpose language capable of also satisfying the requirements of current Elm users. If it is possible, I would rather support Evan as a contributor than some other project, including my own, that expands the work that he started.
Correct me if I am wrong, but I don’t see that not allowing reference cycles and not allowing lazy data structures are necessarily related: Memoized lazy data structures can contain cyclic references or not, depending on the algorithm where they are used. In fact, current Elm still allows lazy deferred calculations, just that lacking the Lazy module (and no means to create one that can be published without being allowed to use native code in modules) the results of the calculation cannot be easily memoized so no true equivalent to a lazy list as can be created as can in other functional languages; Cyclic references come about from structures such a lazy lists (memoized or not) referring back to elements that have already been calculated towards their head. So the possibility of memoized lazy lists seem good to me; cyclic references not often necessary as better algorithms often do not require them.
Not sure what I think about lazy going away, but I’ve never missed it. Then again, its also not a technique that I have ever used much, so perhaps I don’t miss it simply through ignorance.
Yes, that is a bit why I “faded away” in contributing directly to the Elm ecosystem: the sense that (not including you) the general feeling was, if the current main users haven’t found a use for it (mostly meaning a use by No Red Ink) then it likely isn’t important enough to keep. Of course, that is a self fulfilling idea, as if one has to fight to keep a feature, the effort to keep it eventually kills the urge to use that feature (or even that language). Using lazy memoized lists of arrays (or other data structures) is often a way of efficiently and elegantly expressing sequences of complex computations; Using the Lazy facilities of HTML in VirtualDOM is kind of a subset of this that currently exists, but the memoization is specialized to be done interior to the virtualization.
However, the uniqueness detection optimsation sounds pretty neat to me too.
Yes, this is an idea that has been proven to work and has been documented for some use studies to reduce the number of runtime reference counts to something like five percent on average, which may make reference counting faster than the usual Garbage-Collection techniques while destructing deterministically when memory can be proven to no longer be used.
Thanks for clarifying. I see value in your proposition and interrogation, and I am not trying to shut down the conversation with a “Not gonna happen, please move on”.
I do think there is a mismatch between Evan’s vision what you hope for (I might be wrong though!). Also, given how little he has compromised on this vision so far (despite tremendous pressure), I just wouldn’t expect to get him involved with the language you describe.
We could fork the new C backend as soon as it is published, to add those features. But this scenario is the reason said work hasn’t been published. Evan hopes for a payoff for this project, he needs financial stability.
To be fair, I think Evan has chosen a set of principles that make it hard for him to find support: he doesn’t want to work for free (which is fair) and he doesn’t want to change whatever he is building for the sake of founders/donators (which is a deterrent to many).
Now, at the risk of going off-topic : Have we considered a crowd-funded, no-string attached “grant” to Evan to execute on whatever his vision is? He seems a brilliant man and a kind soul, and has already produced something of tremendous value. We would then be the ones making the compromise: we’d be investing in the pure FP ecosystem in general and not necessary in a tool that ticks all our boxes. And maybe making not so much an investment with direct return as a gesture of gratitude to Evan.
Like a Patreon Page? I find this idea really funny XD. But isn’t the Elm Software Foundation exactly to make it possible to donate?
I believe that Evan would probably need a few companies behind him to make his efforts profitable. And I assume he already tried reaching out to any companies that would come in mind.
Edit:
Im not seeing a big “DONATE” button on the Elm page. So maybe that would help?
In practice, there could be value in a one time coordinated campaign and effort from the community, versus donations trickling down from the donation page.
Maybe it could represent a single “action” that we are taking as a community, that we can communicate about, and would result in more involvement and money overall?
Maybe it could result in a decent amount and the Foundation can budget for a given time period, and fund a few of Evan’s projects, rather than doing it month-to-month?
Maybe we get to a milestone we can celebrate, and we will want to do it again later?
I have seen creators (admittedly Youtubers) working for years, not mentioning their Patreon page, and one day they go “I have shot this and this projects and I can’t afford to have the videos editted, if I ever needed your help, it’s now” and response has been overwhelmingly positive. If we can skip the cry for help (though Evan’s talk felt like one at times) and make something nice happen to Evan and Elm development, that would be something to be proud of.
Actually, yes. Just promise me that my lack of free time, competence and experience with these matters won’t be held against me while I make the attempt.
By self-hosting, I assume you mean a compiler for Elm written in Elm itself. A lot of work has already been done in this direction with the elm-in-elm compiler, so that would make a great starting point.
A potential drawback of this, is that I know the type checker in the elm/compiler uses in-place mutation for speed purposes.
But perhaps with the reference counting and auto uniqueness stuff, a version of the algorithm in Elm might get the same level of optimisation from the compiler, and then self-hosting would not have this potential performance penalty.
If doing some hacking on Elm compilation, I would just start with the Haskell compiler.
Perhaps trying out the type checker algorithm in Lean4 would be an interesting project to benchmark agains the Haskell version.
That’s our problem that I’m trying to define in this post. Other than the talk linked in my OP, which primarily explained the pros and cons of different funding models for language development and only secondarily mentioned his new project, he hasn’t really given us a clear vision of where he sees us going. In his talk related to the new project, he seems to be more open to outside influence on features and implementation than he was/is with Elm and hopefully more receptive to contributions. Part of the reason for the stagnation of the actual Elm core and compiler is that it seems that almost no one handles PR’s from contributors and almost all of the accepted ones need to be personally reviewed and accepted by Evan rather than able to be reviewed and accepted by a group that abide by the contraints that he had defined as BDFL.
We could fork the new C backend as soon as it is published, to add those features.
Of course we could, or we could fork the Elm repositories and do it all ourselves, but that would be the least preferred route as he is the inventor of the Elm syntax we love and want to see extended to where it deserves to go; which is the point of this thread.
To be fair, I think Evan has chosen a set of principles that make it hard for him to find support: he doesn’t want to work for free (which is fair) and he doesn’t want to change whatever he is building for the sake of founders/donators (which is a deterrent to many).
Exactly, in other words than what I have used. Although the Elm ecosystem is not dying with contributions from many current Elm users. Elm development is stagnant without any releases for years. A slow release cycle of a year or so would be acceptable - no releases for four years isn’t really acceptable and we only hang in there because we love Elm.
Now, at the risk of going off-topic : Have we considered a crowd-funded, no-string attached “grant” to Evan to execute on whatever his vision his?
Thank you for this suggestion, which is kind of what I hoped for out of this discussion.
He seems a brilliant man and a kind soul, and has already produced something of tremendous value. We would then be the ones making the compromise: we’d be investing in the pure FP ecosystem in general and not necessary in a tool that ticks all our boxes. And maybe making not so much an investment with direct return as a gesture of gratitude to Evan.
Yes, I think we all feel this way. Looking at the development over the years and his various published talks, I think Evan has gotten himself into a trap for which his only solution has been to continue with his hosted funding, whether that be for the Elm-as-it-is project or for the Elm-outputting-PostgreSQL project. Your proposal might free him from that rut and allow him to pursue what he really wants to do.
The thing I feel obligated to point out is that we as investors in a crowd-funded Evan take a risk that he is unable to resume anything close to his initial productivity (an incredible number of completed projects including all of the core packages and the evolution of the Elm compiler) over the course of about four or five years, and/or continues to invest his time only in niche projects such as generating PostgreSQL that is likely not of interest to the general population.
However, as you say, this is also in the spirit of a gesture of gratitude to Evan and the potential positive outcome is perhaps worth the risk of not progressing as we would like.
Shouldn’t we get some sort of response from Evan to this proposal as to what he would like to do if released from financial obligations? Crowd-funding works very well for some major language developments as in Nim (which development has gone on for many years) and the new V language (whose goal seems to be a better Go language), etc.; however, those successful projects also have a well managed system of contributors through a core team which must needs be adjunct to funding to be successful.
Nothing and nobody says you have to do it entirely yourself. Your offer to head the effort is appreciated, but I would hope that if you run into problems, others may contribute what you lack. I would definitely support you, and don’t have a problem with free time, but competence and experience in these matters would have to come from others, who I’m sure will step in.
By self-hosting, I assume you mean a compiler for Elm written in Elm itself. A lot of work has already been done in this direction with the elm-in-elm compiler, so that would make a great starting point.
I’ve looked at the elm-in-elm project but fear it will never be completed as it is a “from scratch” effort and viewed more as a learning exercise. There is much to be learned from Evan’s Elm compiler in Haskell and much of that can be directly applied to Elm, although there are a few performance killing problems with current Elm in lack of linear arrays, concurrency, and ST-type monadic mutation chains.
A potential drawback of this, is that I know the type checker in the elm/compiler uses in-place mutation for speed purposes.
Yes, it uses IORef a for in place mutation in one smallish UnionFind module in the Type Inference section which would need to be replaced with something less efficient that doesn’t do in-place mutation.
There is also a problem in that the Haskell data graphing package used by the compiler also uses mutation of linear arrays for efficiency and a translated Elm version using persistent data structures must needs be less efficient.
However, a self-hosting Elm compiler, while useful for small projects such as a REPL’s/IDE’s not requiring a server, has limited practical use for larger projects; however, it would be a means to an end in development of a general purpose Elm…
But perhaps with the reference counting and auto uniqueness stuff, a version of the algorithm in Elm might get the same level of optimisation from the compiler, and then self-hosting would not have this potential performance penalty.
Yes, once Elm generates C with optimized reference counted memory management and has a couple of extra core packages allowing “pure” mutation including linear arrays, the few inefficient stop-gap modules would be re-written to be at least as fast as the current Haskell versions.
