This is a follow-up to a previous topic I can no longer reply to. My question in short: If we can’t use private packages, how do people collaborate on packages? The long version:
I have an app that uses some third-party packages.
What often happens is that I want to fix a bug or add a feature to those packages.
In other developer ecosystems I would fork the package, make changes, try them in my app using the fork and then submit the changes back upstream to the original forked package.
The try them in my app part is crucial, the changes are seldom so simple as to make them without testing in a live app.
But since we can’t use private packages in Elm, there’s no easy way to test those changes in my app, is it? (That’s the point of the previous thread I have linked to.)
Some changes can be tested using unit tests, but often that’s not enough, maybe I need to verify that the changes work against some backend service.
In other developer ecosystems I could write an automated end-to-end test to verify the changes against a live service. In Elm it doesn’t seem feasible to use side effects in tests?
In the end, the current Elm ecosystem makes it harder for me to collaborate effectively on packages. Or am I missing something? What’s your workflow?
One thing you can do is fork, and then add your fork to your project as a git submodule and add the submodule to the list of source directories in elm.json. Like this, you can iterate until you have made the update that felt right for your use case. If you want to contribute back you can contact the author of the original package and ask if they are interested in your update. Unless if those are too specific, or maintainer does not have time, people are usually happy to get feedback with a concrete use case to improve their package.
The above solutions work but are a bit of a hassle. One major short-coming is that the transitive dependencies are not dealt with when you put a source path to another package in your elm.json; you must also install any transitive dependencies too.
I think something like elm-github-install really needs to be brought back. Perhaps a better name for it might be elm-git-install. In addition to the elm.json file there would be a second elm-git.json file, that provides optional overrides to dependencies, either to a location on the local file system, or some git repo (+ optional tag). There would be a command elm-git install, that does the necessary installation work based on the contents of elm.json and elm-git.json and more gracefully handles the transitive dependencies.
This would make it much simpler to work with forked but not yet re-published packages, to fix bugs in core packages and test them, and for larger software systems to make use of private packages that should be kept internal to some organisation and are not intended to be published.
No, as that only installs the packages without their transitive dependencies.
Also, that did not support the local filesystem or a git branch as an install source. Local filesystem is very useful if you were working on fixing a bug or doing something experimental in say elm/virtual-dom - you might want to test changes you made without pushing back to your fork, or you might want to test a branch without a tag during development (you can use the git SHA at least).
Looks like it might be worth trying - but I’d prefer if the location overrides go in a separate file, in case some other tooling is using elm.json and objects to it being extended.
I also forgot that the name elm-git-install has already been taken. Other names for my proposed tool might be elm-git or elm-local or elm-private.
Why is Martin’s reply flagged and hidden? It’s certainly not abuse, it’s useful and adds to the discussion. (I’ve tried to notify moderators about the false flag, but nothing happened.)
I’m guessing it’s because the tool supports writing packages containing JS and the readme contains the line
Note: Best not talk about this on the offical Elm channels unless you’re trolling.
To be clear then, I linked it because I thought it might support @rupert’s use case and I suggested elm-git-install first to avoid unnecessarily promoting tools that promote writing packages with kernel code.
Just to be clear, I am not advocating for a tool that lets you write and share new packages containing kernel code. I would like a tool that supports 2 use cases, and I think these do not in any way subvert the purity of Elm packages:
Private packages (not containing kernel code).
The ability to modify packages that do contain kernel code and build them locally. This is to assist with bug fixes or other feature experimentation. It would not be possible to publish these as they would need to be in the elm or elm-explorations namespaces. Technically it would be possible to share them as any override to the git URL would be supported by this tool - but I think end users would be very aware that they are using forked kernel packages. I think not being able to publish them to the official repo would make them irrelevant anyway.
I’d sooner have use case 1 over use case 2. Perhaps use case 2 has a better solution?
For example, TSFoster and myself collaborated on a fix for a non tail recursive function in the debugger that is occasionally blowing the stack on larger models. I never figured out how to test the patch though, so have never created a PR for it. Seems a shame to miss out on little bug fixes like this for lack of a simple way to test them: