Revive deleted code of Elm packages

Yesterday @malaire deleted the GitHub repositories to his 2 Elm packages malaire/elm-safe-int and malaire/elm-uint64, which meant we suddenly couldn’t build Airsequel anymore.

@malaire I understand that you’re apparently on longer interested in Elm, which is fair enough. But was actively sabotaging the package ecosystem really necessary? There are a lot of Elm developers who’d be happy to take over the repositories for you, if you don’t want to deal with them anymore. I’d highly appreciate it, if you could help us restore them! They were valuable additions to the ecosystem and it would be a shame to lose them just like this.


I was actually quite shocked that it’s that easy to just completely remove dependencies and that there is no backup / fallback if the primary source location vanishes.

Is there a way to improve this in the future?
E.g. mirroring all repos referenced by packages to a dedicated elm-packages-archive organization and loading the code from there?

5 Likes

I didn’t just delete those two repositories, I deleted all my Elm-related repositories and relevant sections of my website because I don’t want anything to do anymore with this buggy dead language.

But was actively sabotaging the package ecosystem really necessary?

I wasn’t sabotaging anything. It’s not my responsibility to keep my repositories available indefinitely especially when language is so buggy that its core developers don’t even want the relevant bugs mentioned publicly but prevent people from reporting bugs.

Also by having any Elm-related repository available (or anything Elm-related on my website) I’m (indirectly) advocating for Elm.

And I just can’t do that anymore given the current condition of Elm.

I was actually quite shocked that it’s that easy to just completely remove dependencies and that there is no backup / fallback if the primary source location vanishes.

This is what happens when you use buggy, badly designed, unsupported, dead language.

Is there a way to improve this in the future?

No because Evan doesn’t seem to want anything to improve.

4 Likes

btw, I know well over 100 Elm packages which doesn’t work anymore because of changes in their GitHub repository. This has been a known issue for years but Evan just doesn’t seem to want to fix it.

(Elm compiler is hard coded to use https://package.elm-lang.org/ and via it the primary GitHub repositories - changing this without forking compiler is possible - I’ve done it - but not trivial.)

Anyone using Elm should have complete backup of all Elm code they need, with knowledge of how to recover from situations like this, since these happen often.

2 Likes

You should be able to recover the source if you still have it in your ELM_HOME/0.19.1/packages folder. As Elm requires all packages to have open source licences, you can then re-publish them according to the licence conditions under a new name, ad-si/elm-unit64 for example. Or you could just copy the code into your application and not re-publish it.

Let us know if you do not have the sources any more under your ELM_HOME/0.19.1/packages folder, some one at least will have them, and me can make a .zip file for you to recover them from.

I think elm-unit64 in particular is worth preserving, so if no-one else recovers it I shall see if I can find it.

2 Likes

I’m in the process of republishing both packages. Should be ready in half an hour.

10 Likes

Nice one Martin. :+1:

And no hard feelings Malaire, there is no compulsion for you to stick around here if you are done with Elm.

10 Likes

On the Elm dependencies analyzer - this tool was useful to me a number of times when my elm.json got messed up:

Are we going to miss this (haven’t used it for ages now)? Or is this now stuff that elm-json can do?

I’ve republished both packages now.
https://package.elm-lang.org/packages/MartinSStewart/elm-uint64/latest/
https://package.elm-lang.org/packages/MartinSStewart/elm-safe-int/latest/

11 Likes

Regarding the dependency analyzer. It’s not an online tool, but if you need, you can use Releases · mpizenberg/elm-solve-deps · GitHub to analyze your dependencies. It’s a small example binary I made to showcase usage of the rust elm-solve-deps package, which is also the one converted into wasm and now used in elm tests.

3 Likes

Thanks for the emergency response, people.

Is there any chance we can address some of the strong things that @malaire said? Even though it’s apparent Malaire’s angry, the underlying points seem very much worth talking about, don’t they?

4 Likes

These have been talked about many times. There is no acceptable solution.

If you are talking about relying on primary GitHub sources without backups, then I had a working solution to that: A (local) package server which caches all files so it doesn’t matter if primary sources disappear.

Nobody (except me) was really interested in that because it requires either a minor patch to the compiler (to allow other servers than “package.elm-lang.org”) or local CA with fake HTTPS certificate for package server which announces it as “package.elm-lang.org” and a cache which directs requests for package.elm-lang.org to this other server instead (i.e. this is practically a HTTPS MITM attack put to good use).

If you are talking about fixing other bugs/problems in compiler/core/… then that requires forking compiler, again a thing nobody seems to want to do.

3 Likes

I have this mirror running:

Its in a bit of a broken state right now, since the domain name expired (deliberately, I meant to change it), and it currently still needs a manual trigger to update it and fetch the latest packages.

I feel motivated by this ‘incident’, and also recent threats from GitHub to change .zip checksums, to try and get this to a properly functioning state, just hard to find the time right now, so we’ll see. The infrastructure is open sourced, so others are free to try too: GitHub - eco-pack/eco-server: Alternate Elm package server

That would solve the package deletion issue, since this package server keeps a copy of the package .zips locally, and does not rely on GitHub. At the very least, I should be able to recover old .zips from there, even if it is not used to actually compile against.

buggy dead language

The other issue the Malaire brings up seems to be that Elm is a “buggy dead language”. Really there is no need to bad mouth Elm in this way, and hopefully Malaire will now be leaving us alone to continue working in a friendly and constructive way, since he no longer has any remaining interest in Elm.

There are bugs, and I have spent some time cataloging and documenting them (here). All software systems have bugs, and viewed amongst all open sourced code in the world, I would hazard a guess that Elms bug count/line is at the low end. It is very high quality code, or it would not still be viable unchanged for 4 years. Things are not perfect of course. Anyway, we already have a thread going for this discussion so perhaps better if we just continue the conversation there instead: Request: Elm 0.19.2: any update to help adoption to prove that Elm is not dead?

Its not dead, or there would not be people like Martin and myself prepared to put in effort like this on “reputational damage limitation” or keeping the package eco system going and so on. Not that we want to be doing things like this, but there are probably about 20 people who have stayed solid with Elm over the last 5 years that are all competent engineers, and who will do what is necessary to keep it going - mostly we are just busy with our real jobs.

So please, if you used Elm in the past and enjoyed it, don’t get angry and pull the plug. It isn’t necessary and it costs you nothing to continue to be a friend, and to leave your Elm packages intact on GitHub. Or if you really want to remove them, just come on here and ask if anyone is using them and perhaps if they would like to take over maintaining them.

9 Likes

And that list is incomplete, I know of at least one runtime exception not on that list. But then Elm “collaborators” don’t want bugs listed which is why the relevant issue is locked and nobody ever updates it with new information.

ps. And sure I’ll stop visiting this forum. I’d even delete my account if that were possible (it’s not).

2 Likes

I get the impression it was locked because it was considered to be:

a) complete as far as a particular phase of work was concerned
b) unhelpful to have such a list that keeps growing
c) some other reason relating to how Evan wants to organize his work

Your assertion that “its core developers don’t even want the relevant bugs mentioned publicly but prevent people from reporting bugs” is simply untrue. Anyone can open a new issue on GitHub. I even remember reading a note from Evan (can’t find it again, sorry) asking to open a new issue for any new problem you find, even if there is a duplicate already. Given that guidance, if you have some runtime exception issues to report, please do go ahead and open a set of new issues for them, there is no need to maintain a separate issue to list them all - they are already in a list as GitHub issues.

Yes, my bug list is incomplete. I was really trying to focus on just the bugs that have PRs against them, plus a few more that people asked to have added to the list because they felt they were important. The purpose of that list is to drive an independant patching effort elm-janitor.

That effort led me to examine the bugs that Elm has more closely, in order to try to understand if the bugs are the real problem that Elm has, and how severe it is. My conclusion is that it is not terribly severe, and most things seem to either have workarounds or are low impact. The real issue seems to be one of perception and also a lack of visible leadership and the lack of possibility for active engagement in furthering development of Elm compiler+core from its community (bus factor, “real” open source, etc).

5 Likes

I think we should start new threads with particular focuses (e.g. what do industrial users want for marketing support? What do industrial users want for recruiting and hiring support? what do job seekers want for job finding support? What libraries do industrial users want? What is the current pain level with open bugs in compiler? Etc) instead of running that one on for much longer, but it would also be nice to just have some friendly elm code organizations discussions before getting back into the unpleasant topics.

1 Like

100% agree with this, we have found work arounds for all issues we came across. The real issue for adoption is the poor perception of the language leadership, which I think is deserved unfortunatelly.

For me this perception can be summaries in two points

  • No interest in mantaining or improving the language and ecosystem
  • Discouraging attitude towards people that want to do the above

I really wish the perception of Elm would change as I like working with it a lot. Unfortunatelly the lack of delegation is a big problem with Elm. Unsure what we can do, we need to encourange, accept and collaborate with people that want to improve things.

4 Likes

I want to push back on one thing you said: I’m pretty sure Evan is strongly interested and actively working on improving the language. He just has his priorities so that bigger long-term improvements win out over the smaller more visible ones.

Otherwise, I think this is pretty accurate.

1 Like

Yes, exactly and I think we should avoid Evan bashing. The way I see it is that he has given us a huge amount, it seems ungrateful to complain that it isn’t perfect. He has a new baby too, few things in life mess up your priorities as much as that.

I really think it doesn’t matter, as there are ways that we can work around him. An analogy might be Linux and Linus - 3rd parties had to come in and start packaging Linux into distros to make it usable, and its fine that Linus was dedicated to working on the kernel (and being rude to people). I have proposed this idea before, which is that we create a maintenance fork for the purpose of tidying up bugs.

4 Likes

And actually, people DO fork the language. Look at gren, lamdera and elm-pages v3 (yes the last 2 use a modified compiler) and surely plenty of others.

Yes.

But I would also like to point out that Gren is not backwards compatible with Elm.
Lamdera (elm-pages 3 uses it) is backwards compatible. But it also adds new features, so if you start to rely on those there is no simple way back to Elm. Its not forward compatible.

Which is all fine. But also I am looking after a 150K LOC commercial Elm project, so these distinctions are important to me at least.

I am proposing a “maintainance fork” by which I mean something that remains 100% compatible. It could eventually become a “feature fork”, but I would prefer to wait a while before taking that decision. That decision would come after all the bugs that can be fixed without sacrificing compatability have been fixed (that was the purpose of analysing and ordering the bug fixes Elm Essential Bug Fixes), after sufficient time has passed to see if there is going to be another Elm release or not, and a large time window had been allowed for others to use as an on ramp to the fork.

On 100% compatible front, you could argue about is something a feature or a bug right? But hopefully its usually clear when something is a bug and not a feature. But that is also what makes doing this only work as a group project, because it is not just for me to decide that. Any bug that I want to patch, I feel it must be run past some other sets of eyes first. That is what I used the #elm-janitor channel on Incremental Elm Discourse (https://incrementalelm.com/chat/) for and people were good about reviewing stuff there.

6 Likes