A process for core library fixes and improvements

[mattpiz]

how they solve the core maintenance problem?

It very much depends on the personality and social skills of the language leaders and is also tied to the funding model of the language, the two being intertwined together.

You can see a discussion between Evan (Elm), Jeff Bezanson (Julia) and José Valim (Elixir) about funding a language here on Jean Yang’s channel: https://youtu.be/f0ouXATi25I

As hinted there, what works for some is different than what works for others. Collaboration at the scale of a widely used programming language requires a lot of communication, even just for maintenance and bug fixes. And as Evan says it himself in the video, it is not where his strong suits are. Which I’d say are rather around overall language design and technical achievements. So it’s understandable that he wants to spend his time there instead.

Back on the process regarding possible improvements. I don’t think we should restrain ourselves to not do anything, just by fear of community fragmentation. If what we try to achieve tries to stay as close as possible to elm but improves user experience (like @rupert approach), this will ultimately be beneficial and bring new people instead of fragment the current community.

That being said, all these initiatives are individuals from the community spending some of their free time. So “Elm is gonna stay as it is now for quite some time” is a fair assessment of the situation in my opinion because progress of individuals on their free time is something measured in months rather than weeks. At least as long there isn’t a company with the will and financial capability to support employees on the matter.

[berend]

@rupert

I have not yet run into a showstopper bug with Elm, only minor annoyances.

True, but so will everyone doing Elm at some point. In the end you end up with thousands of people running into minor annoyances again and again and again.

A light fork and fork of core packages would allow more people to contribute to Elm.

[berend]

@mattpiz

If what we try to achieve tries to stay as close as possible to elm but improves user experience

Exactly. The point is not to improve the language, as few of us would have the skills and the vision, but just address the issue of patches lingering for years, and make it possible to create core libraries for many other areas where Elm can be used.

[Janiczek]

I’ve edited my original post to add a summary section to new folks coming to the thread and not wanting to read through all the replies. Hopefully I’ve caught all the important points and nuances. If I’ve taken some things out of context please correct me.

[ronanyeah]

+1, I’m not that interested in API changes or performance improvements, but I would fully support and use any fork that is willing to address longstanding bugs.

At this stage I don’t think it possible for one person to manage the core libraries, and change is required and inevitable.

[jhbrown]

@Janiczek

The question is, while this is a cleaner solution than eg. patches applied by tooling on Elm libraries, do we have enough resources as a community to make the jump?

It’s hard to answer this question in the abstract. I think it might be helpful to define a specific task set – both the one-offs and the ongoing maintenance taxes – and estimate the associated labor, again both one-off and recurring. This isn’t just technical work, it’s also setting up a foundation (?) or the like to own copyright and take donations and pay for servers, getting a new logo designed, and so on and so on.

If it’s something 2-3 people can get started in a month dedicating one evening each week, that’s different than if it’s going to take 3 FTEs 6 months to get it all put together. Same story for maintenance.

[gampleman]

One option floating in my head was to start a small business that would fill this niche. Basically it would provide an SLA for bug fixes requested by its customers plus perhaps private packages as the core offering. This seems like it would be quite beneficial to the Elm community, since it would provide business types with a simple answer to some of the maintenance questions in this thread (i.e. simply throw some (predictable amount of) money at the problem).

However, I have no idea how sustainable such a model of development could be. So here is a quick poll on that question:

Would you (or your employer) be willing to pay for a commercially maintained fork?

• No
• Yes (at most $5/developer/month)
• Yes (at most $45/developer/month)
• Yes ($100/developer/month or more)
• Yes (on time $1000 per developer)

[rupert]

If I think of starting a business around something, I also think, is this the business I would start if my aim was to succeed by making money?

I voted that my company will pay £0 for a maintained Elm, because I don’t think it is the best way to manage things for the benefit of all.

My reasons are:

• I feel slightly demotivated to contribute to Elm, pureley out of love for it, knowing that a business that doesn’t give freely back to me is consuming it for profit (if you are so lucky, I don’t think you will make money anyway).

AND

• Such a business would inevitably act in its own interests, whilst wielding influence over something that matters to me.

I don’t in any way disagree with businesses making money from Elm (if they can) from the position of being a more neutrally oriented third party, not one that controls its future maintenance and direction.

I think contributing to Elm as a package author has thus far been very rewarding. In addition to really enjoying writing those packages, I also know I have greatly multiplied the effort in what I receive back. That kind of positive cycle is what I would most want to retain about it. I also think it is the way that is most beneficial to all.

I think we are slowly working towards a fork, but I don’t see it as urgent or the next thing that should happen, since there is still a lot of scope to make improvements before that really needs to happen.

An idea I have put forward before is that a fork of it could become an Apache Foundation incubator project. That has some potential advanges:

• Apache has been around a good while and is generally well respected.
• They can provide hosting resources.
• It can help with getting corporate sponsorship and act as a conduit for funding.
• They are experienced at mentoring open source projects.
• It helps to establish a good governance model for the long term management of a project.
• Bus factor has to be resolved before a project can graduate from the incubator - all top-level Apache projects have multiple contributors.

[gampleman]

I’ll keep to poll open until the thread closes, but it seems that there is some interest, but (as I suspected) probably not enough to even pay a single person half-time.

---

I feel slightly demotivated to contribute to Elm, pureley out of love for it, knowing that a business that doesn’t give freely back to me is consuming it for profit

That’s not necessarily how that would work. I think a company set up like this would generally work as open-source and potentially (if the core elm team wanted) could even send PRs to the core repos. The thing one would pay for isn’t exclusive fixes, but the knowledge that someone has your back if a show-stopping bug appears in the core library as well as easy tooling to apply those (and I suspect making that tooling seamless would be a big bulk of the work, especially as few people use the elm compiler just on its own, but rather through integrations with Webpack et al).

Such a business would inevitably act in its own interests, whilst wielding influence over something that matters to me.

Not clear to me that anyone has much influence around here outside the core Elm team. If this hypothetical company became so indispensable that it would start getting influential despite that, I think that would be a very strong signal (and note that Evan and company seem to be very interested and sensitive on the topic of corporate influence on open source) that the current maintenance model may need to be reconsidered. Conversely, if such a company went quickly out of business (or languished in obscurity), it would provide validation to the hypothesis that things are fine as they are.

---

I don’t think you will make money anyway

Well I wasn’t planning to be the one to run this, but I agree that it doesn’t seem to be an amazing business plan as things stand.

[lucamug]

An “Elm 1.0, Corporate Edition” could be useful for marketing reasons, to facilitate adoption.

Other than that, for the way we use Elm at work, I don’t have any strong complaints/requirements about the present Elm implementation.

Edit: I was correctly reminded that, if this is a fork, the name need to be different from “Elm”

[Maldus512]

@lucamug

An “Elm 1.0, Corporate Edition” could be useful for marketing reasons, to facilitate adoption.

Remember that the name should be different. I’m not sure how much of marketing benefit would be left while having to create an entirely different brand.

[nil]

There are sooo few bugs in the excel doc linked above here for a language that is in pre-release where the latest release has been out for almost 2 years. It would be nice to have an effort to squash them (or at least try, and if its complicated defer it.). Even a “go” from a maintainer that “PRs for these are welcome, and we will review them in N weeks” would be a nice solution.

[Janiczek]

I believe the need to differentiate is in name and logo.

Let’s say the fork is named Frob (horrible, I know). In my mind the Frob webpage could mention that it’s based on Elm 0.19 or that it’s a fork of Elm 0.19 with focus on X, Y, Z…

[gampleman]

Surely the natural name would be Elf Could be even a backronym for Elm Language Fork

[rupert]

Eco - Can stand for Elm Compiler Offline. That is, its the elm compiler with the built in packaging stuff removed into a separate tool (eco-install). Also about opening up the Elm ecosystem.

[rupert]

@gampleman

That’s not necessarily how that would work. I think a company set up like this would generally work as open-source and potentially (if the core elm team wanted) could even send PRs to the core repos. The thing one would pay for isn’t exclusive fixes, but the knowledge that someone has your back if a show-stopping bug appears in the core library as well as easy tooling to apply those.

I think we are generally on the same page on this.

One complication and part of the reason for my viewpoint, is that currently we cannot send PRs to the core repos because the BDFL is not engaging with this (and has given his reasons). So a company maintaining Elm would necessarily have to create its own package repo system. For example, supermario mentioned that Lamdera could probably already take on that role, since he pretty much had to solve that already.

I am not against business paying for fixes, or having employees or consultants with commiter rights on the open source projects they consume. That is very often the case with Apache projects for example - many AWS services are obviously bundled and branded Apache projects. Another example of a company that consumes many of them is RedHat. They definitely have contributors on the projects they are interested in, but the governance model of Apache does not allow them to exclusively capture a project.

I think if you work as an Elm consultant or a web-dev business, it can be a good selling point if you can tell your customers that you are involved in core maintenance, have their back, can get critical fixes in if the need arises. Selling that as a service might be a good source of occasional side revenue. Or it might get you onto a project where you are expected to play that role if the need arises.

But I still think we would need the neutral, democratic, collectively run ownership of the core being maintained sitting outside of any particular business operation.

[jfmengels]

@Janiczek

3. make tooling to easily apply patches to packages or use forked packages

I think that if the community were to do be pro-active in solving the mentioned issues, this is likely the way to go, as this is the least diverging or obtrusive solution.

From looking at the Google sheet link with the list of issues, most issues are located in core libraries, not in the compiler. Other issues (like 2 ^ -1 maybe?) could be patched by altering the compiled code. A combination of the 2 would resolve most problems I think.

Forking the compiler does not seem that “necessary” to me, and it would by far be the most arduous task as well, and I don’t think it’s cost-effective to do so.

---

On a separate note, as a tooling author, I am quite worried about the directions this may go. I don’t think me and other tooling authors want to support both Elm and a fork (or several forks), as that could be a lot of work for us, and each individual author would need to be convinced individually.

If some decide to fork Elm, my advice if you want to continue using Elm tooling (IDEs, elm-review, …) is to not change anything about Elm’s workings. For instance (but not limited to):

• Don’t change the way the Elm CLI works
• Don’t create an alternative elm.json nor change its behaviour
• Don’t add/change new APIs to core modules
• etc.

Most tools out there already support passing an Elm binary as an argument, so an alternative compiler would be very easy to support, as long as it works almost exactly like the Elm compiler.

That is why I am thinking that applying patches to the packages and altering the compiled source code is probably the way to go. Users still get to benefit from Elm tooling, tooling authors don’t have to spend time supporting a fork. And also, we probably then don’t need to change the name of the fork because there wouldn’t be a fork, just an additional tool to point people to. elm-optimize-level-2 kind of already does this (note that this would not be the tool to do this), and it didn’t have to fork the project.

I would urge whoever that feels like taking up these task to change as little as possible to Elm in their quest to solve the issues mentioned in this thread.

[Janiczek]

Just to further the points in 3. a little, in addition to @pdamoc 's proof of concept with patching ELM_HOME linked above I’ve started dreaming up a format of how these “patched” snapshots could be described declaratively and applied.

I’m sure there is a bunch of other prior art in all of the elm-git-install and shelm-like tools that we could explore.

Here’s my noodling:

elm-patchwork.json

{
  "elmVersion": "0.19.1",
  "packages": {
    "elm/core": {
      "version": "1.0.5",
      "base": {
        "gitRepoUrl": "https://github.com/elm/core",
        "gitCommitHash": "84f38891468e8e153fc85a9b63bdafd81b24664e"
      },
      "patchPlan": [
        {
          "name": "List.concatMap performance improvements",
          "filePath": "./patches/elm-core-1.0.5-list-concatmap.patch"
        }
      ]
    }
  }
}

consumed by some tool into types like:

{-| The top-level type corresponding to the elm-patchwork.json file
-}
type alias Project =
    { elmVersion : String -- "0.19.1"
    , packages : List Package
    }


type alias Package =
    { name : String -- "elm/core"
    , version : String -- "1.0.5"
    , base : PackageBase
    , patches : List Patch
    }


type PackageBase
    = BaseFromGit
        { gitRepoUrl : String -- not strictly GitHub?
        , gitCommitHash : String -- this is how we'll download that package
        }
    | BaseFromFilesystem
        { folderPath : String -- "./patchwork/bases/elm/core/hacked-1.0.5"
        }


type alias Patch =
    { name : String -- "List.concat: performance improvement"
    , contents : PatchContents
    }


type PatchContents
    = PatchFromGit
        { gitRepoUrl : String -- not strictly GitHub?
        , gitCommitHash : String -- this is how we'll download that patch
        , filePath : String -- "patches/elm-core-1.0.5-list-concat.patch"
        }
    | PatchFromFilesystem
        { filePath : String -- "./patches/elm-core-1.0.5-list-concat.patch"
        }

[jerith]

I did encounter a show-stopper bug in elm/core in my app in Jan. 2019. It was fixed in Nov. 2019.

After waiting about a month for my PR to be merged, I began applying the fix in my nix build script. When the fix was eventually merged, I updated and stopped patching.

Should I encounter any other show-stoppers in elm/* code in the future, I plan to use a similar approach. The main downside is that the fix didn’t kick in during development, only in “real” builds. (In this case, that was fine, I could just avoid the known landmine of albums containing more than 100 photos during development.)

[berend]

@jfmengels

Forking the compiler does not seem that “necessary” to me,

I think nobody wants to change Elm, but would like to use Elm in more cases. Supporting certain JavaScript browser APIs requires the ability to use JavaScript more directly. I.e. it would be good if you could write core libraries more easily to allow for experimentation.