A process for core library fixes and improvements

Do you know if there is any elm/compiler community fork and what it offers?

And Evan wants that to happen. He isn’t against forking.

I’m sorry, but I could use some further explanation on this claim. I have been following the Elm scene for the last four years (checking out the forums, watching Evan’s talks, reading the few articles around, experimenting and using Elm for some small projects) and I can’t say I get that impression.

Beside one comment - that I’m too lazy to find and link - were Evan says that he wouldn’t be opposed to a fork as long as it used a different name (as stated by the BSD-3 license), there hasn’t been much “official” talk on the topic. On the other hand, it seems to me that actions and policy around the Elm community give the exact opposite message.

For one, the development process is completely closed to a restrict circle of developers. It’s hard to create a fork when no one is allowed to contribute to - and therefore has experience with - the original project.

The decision to lock some language features behind package DRM goes in this direction as well: powerful tools like native modules would help in experimenting with the limits of the language before resorting to a compiler fork. I find this issue (int parser fails on trailing decimal separator · Issue #14 · elm/parser · GitHub) for elm/parser exemplary of this: any community made number parsing package (created to fix a couple of ignored issues) would always be a second hand alternative due to custom operators being a privilege for inner Elm developers.

All of this raises the bar in the effort required to go through with it and dissuades potential maintainers.

Finally I can quote a single attempt (or rather, the serious discussion of) to fork the compiler that followed version 0.19: Support for elm 0.19 · Issue #62 · gdotdesign/elm-github-install · GitHub
It is a discussion of a possible fork that is abruptly shut down by a prominent figure in the Elm community, basically trying to bully these developers into desisting.
Now, there is no evidence that the interference actually hindered the effort and I don’t think the person in question is officially affiliated with Elm or Evan, but to the best of my knowledge the latter has remained silent about that whole exchange. I find hard to believe that he did not know about it and it would have been a great occasion to make explicit his favorable opinion on a fork.

I’m sorry if I sound harsh, but given my point of view I have an hard time believing Evan would look positively at a fork.

3 Likes

@Maldus512 I’m fairly certain Lamdera is a fork and a commercial product, which contradicts the idea that Evan is against forks.

Additionally, can we keep this focused on maintenance and not theories of what Evan may or may not think? This thread would benefit from staying on topic and not speculating about the intentions of others.

11 Likes

So in summary:

  1. Having a process for fixes and improvements in the core library would require some involvement from Evan → But we as a community should not depend on him
  2. Forking the core libraries would make us independent of Evan → But this requires forking the compiler
  3. Forking the compiler is hard to do, and past attempts were shut down.

So to continue the discussion:

I’d like to help forking the compiler. In my eyes there isn’t a lot that needs to be changed:

2 Likes

I’d still like this thread to wait for Evan’s response to:

Evan, what are you generally thinking about the “small fixes” maintenance need for elm/core etc. repositories?

Could you please share whether you considered some other-than-batching maintenance model (perhaps in later stages of a project) and what criteria would such a model have to pass? Also, any thoughts on delegation? Generally, how to move this discussion forwards?

2 Likes

Hi everyone!

[have] you considered some other-than-batching maintenance model

He most definitely has, and the current strategy is a deliberate choice. Regarding delegation, then he touches upon the topic in his most recent talk The Hard Parts of Open Source. I recommend watching it if you’d like to understand his position better!

P.S. I think a fork would be great! Given that it is not the first time it has been suggested, it seems at least a few more people would be interested in exploring a more distributed governance style :blush: As already mentioned, just remember to change the name and logo!

11 Likes

Indeed, this fork option has been discussed a huge amount of times (including by you @terezka if I remember well) but never actually shown up.

My interpretation of this is that maintaining a whole language/platform/community is a huge work, asking for a lot of investment (and I hope this previous sentence will be read as a recognition of the amazing Evan’s work quality). One have to considerably want improving/changing the language to see enough benefits counterweighting this investment (like Mario did with Lamdera).

I read Martin’s request as “How could we help fixing this little int bugs in elm/parser reported 3 years ago and fulfill this “TODO” in deadEndsToString?”. I hear the will to “batch pull requests” but here this really are bugs which really give a bad image (“What? You want using a language which has a stupid known bug in its std lib for 3 years and didn’t fix it?”). Forking the compiler to just fix those little bugs clearly doesn’t outweigh the investment needed.

12 Likes

I wrote a spec for this before starting work on it, and this still reflects pretty well what I am trying to achieve in the first version - see the use cases listed at the bottom: Elm Package Server · GitHub

In the interests of being as uncontroversial as possible for the first release I also wrote:

  • Confirm the package does not contain ports.
  • Confirm the package does not contain kernel code.

I do also think, if I get to V1, that I would be interested in being able to publish patch releases to core packages. I have actually patched and produced a couple of PRs for core packages partly as I am intersted in exploring how this might work in practice. My process is to fork the package at its latest published tag, create an issue describing what is being fixed, linking to related issues in the core packages, and tracking its release status and make a clean patch. That is, to keep things as transparent and maintainable as possible, and to always feed back improvements to the upstream projects - whether or not they are ever accepted there.

I think great care needs to be taken when deciding what to patch. Anything that causes a runtime exception (apart from Debug.todo obviously) is fair game. Issues like this one - int parser fails on trailing decimal separator · Issue #14 · elm/parser · GitHub - I am not so sure about. If the patch never makes it back into Elm, then this introduces divergent behaviour in the alternative system, which could significantly affect program behaviour. There is also a simple workaround, to write your own int parser. So maybe some guidelines on what is suitable for patching need to be written up.

I don’t really see this as a fork of Elm, I would instead call it a release branch, where the Elm Janitor tidying up work goes. I have no desire at all to fork the language, its really a great language and I could not do better.

Some other related thoughts:

  • At some point beyond V1 I can imagine having a package repo that allows kernel code, this might be called the ‘unsafe’ repo. The domain tags concept in my spec would be very useful here as the ‘unsafe’ domain could be tagged and excluded (would be excluded by default, you would have to opt-in).

  • I have never yet run into a show-stopper bug in Elms core or compiler. Probably the closest example was the recent Safari for ... in bug, which I note Evan did get involved in the discussion of. As it affects more than just Elm, it seems like Safari should really own that one. The quality is generally very high.

  • I am interested in using Elm outside of web development because I think it has real potential as a cloud language. Also motiviation for the domain tags concept, since the browser domain is not needed there. This is analogous to the platforms concept in Roc that a few people have mentioned recently.

  • I like the name Eco, but not so keen on the ‘pro’ bit, although it does reflect my intention that it is for supporting professional work. Eco was already taken, so I had to add something to it, better suggestions very welcome.

2 Likes

If Elm core packages are patched, can they be reliably installed and used with elm-git-install? I have never tried this with a core package, but if it works it provides a way of patching them without forking the compiler. Could be a path of least resistance to move this forward in the short term.

1 Like

Could be a path of least resistance to move this forward in the short term.

Yes and I really think it is unfortunate. One of elm’s strength is having super nice tooling and easy setup. If we start having those kind of instructions floating around (“oh just add this third party tool, setup this unofficial package”), elm will become way less friendly.

13 Likes

FWIW @terezka’s reply effectively answers the original post’s questions:

Looks like there are roughly three options:

  1. keep the status quo
  2. fork the language and use a different maintenance model there
  3. make tooling to easily apply patches to packages or use forked packages

To me the 2nd option looks very time-consuming; the 3rd option has a proof of concept done by @pdamoc: GitHub - pdamoc/elm-pkg-patch: Simple example on how to patch elm packages

7 Likes

I’m curious – are there community members who are familiar with other languages and how they solve the core maintenance problem? It seems like a hard problem in general. If anyone can offer a bit of compare and contrast, that might be useful as reference.

I think even before going to 3, it is worth assessing what bugs do we want to fix? Building a tool will help to automate this, but I would like to know, what is the demand for the automation and organizational effort? I ask, because as I say, I have not yet run into a showstopper bug with Elm, only minor annoyances. Of course, it is valuable to fix stuff that is obviously fixable, especially where someone has taken the time to make a good PR for it.

So I created a google sheet to collect information on what are the essential fixes that people want?

Just add the description and github link of the issues that are bothering you, or are obviously broken things you can fix, or things that are fixed and have good PRs that should really be merged.

I am happy to do some triaging on this list, if others help make it. The triaging will be to go through the list and look at all the github issues, and fill in some additional fields like - is the issue still current, is it a runtime exception fix, is it a behaviour change but not an API change, is it an API addition, is it an API change, is there a PR, and so on.

5 Likes

how they solve the core maintenance problem?

It very much depends on the personality and social skills of the language leaders and is also tied to the funding model of the language, the two being intertwined together.

You can see a discussion between Evan (Elm), Jeff Bezanson (Julia) and José Valim (Elixir) about funding a language here on Jean Yang’s channel: #PLTalk: PL Funding Panel with the Creators of Elm, Elixir, and Julia - YouTube

As hinted there, what works for some is different than what works for others. Collaboration at the scale of a widely used programming language requires a lot of communication, even just for maintenance and bug fixes. And as Evan says it himself in the video, it is not where his strong suits are. Which I’d say are rather around overall language design and technical achievements. So it’s understandable that he wants to spend his time there instead.

Back on the process regarding possible improvements. I don’t think we should restrain ourselves to not do anything, just by fear of community fragmentation. If what we try to achieve tries to stay as close as possible to elm but improves user experience (like @rupert approach), this will ultimately be beneficial and bring new people instead of fragment the current community.

That being said, all these initiatives are individuals from the community spending some of their free time. So “Elm is gonna stay as it is now for quite some time” is a fair assessment of the situation in my opinion because progress of individuals on their free time is something measured in months rather than weeks. At least as long there isn’t a company with the will and financial capability to support employees on the matter.

7 Likes

True, but so will everyone doing Elm at some point. In the end you end up with thousands of people running into minor annoyances again and again and again.

A light fork and fork of core packages would allow more people to contribute to Elm.

Exactly. The point is not to improve the language, as few of us would have the skills and the vision, but just address the issue of patches lingering for years, and make it possible to create core libraries for many other areas where Elm can be used.

1 Like

I’ve edited my original post to add a summary section to new folks coming to the thread and not wanting to read through all the replies. Hopefully I’ve caught all the important points and nuances. If I’ve taken some things out of context please correct me.

1 Like

+1, I’m not that interested in API changes or performance improvements, but I would fully support and use any fork that is willing to address longstanding bugs.

At this stage I don’t think it possible for one person to manage the core libraries, and change is required and inevitable.

2 Likes

It’s hard to answer this question in the abstract. I think it might be helpful to define a specific task set – both the one-offs and the ongoing maintenance taxes – and estimate the associated labor, again both one-off and recurring. This isn’t just technical work, it’s also setting up a foundation (?) or the like to own copyright and take donations and pay for servers, getting a new logo designed, and so on and so on.

If it’s something 2-3 people can get started in a month dedicating one evening each week, that’s different than if it’s going to take 3 FTEs 6 months to get it all put together. Same story for maintenance.

1 Like

One option floating in my head was to start a small business that would fill this niche. Basically it would provide an SLA for bug fixes requested by its customers plus perhaps private packages as the core offering. This seems like it would be quite beneficial to the Elm community, since it would provide business types with a simple answer to some of the maintenance questions in this thread (i.e. simply throw some (predictable amount of) money at the problem).

However, I have no idea how sustainable such a model of development could be. So here is a quick poll on that question:

Would you (or your employer) be willing to pay for a commercially maintained fork?

  • No
  • Yes (at most $5/developer/month)
  • Yes (at most $45/developer/month)
  • Yes ($100/developer/month or more)
  • Yes (on time $1000 per developer)

0 voters