What would you think about an $ELM_HOME for local packages?

I write Elm for work in an enterprisey environment. We primarily write in-house tooling for other employees. Our repositories typically have frontend and backend folders at the root of the project, with all the Elm code living in frontend/src. We’ve found ourselves writing similar modules for common needs, but unfortunately cannot publish those modules to package.elm-lang.org since we would not be able to publish work done for the company as open-source projects (at least not without going through a lot of trouble).

The source-directories array in elm.json allows you to add additional source directories “outside” your project in order to have local copies of, say a package, but it has a few shortcomings

  1. You’re limited to absolute or relative paths. elm.json gets checked in, so an absolute path is out, and a relative path would require everyone to have the same repository structure (side-by-side) and, if the elm.jsons are to be consistent, the same internal structure as well. This isn’t ideal. We are also “not allowed” to use submodules, so a relative path in to the repository wouldn’t work.
  2. I’m not sure if it does allow environment variables, but if it did, the syntax for that is different between %WINDOWS% and $LinuxOrMac.

Would some sort of canonical environment variable the compiler used to look for source directories fix this? I am aware the compiler caches files in ~/.elm/0.19.1/packages, but I don’t think it’s a good idea to edit that manually.

I would love to know if there’s a standard way to do this and am open to ideas!

2 Likes

Did you see this in the recent Elm newsletter? I think it covers your use case.

===

Actually that looks like its coming in the 0.7 version which is next on the roadmap:

3 Likes

I will certainly have to take a look. Thanks!

@adamyakes Are you able to use Nix at the company?

Packages are really the right means of sharing reusable code, as you point out yourself. source-directories is not.

As @Rupert said, elm-wrap is made to unlock your exact scenario: extracting code from the source tree into packages, publishing to local, team, org, private repositories; install from Git.

As of version 0.6, you can use wrap package extract to turn chunks of code into packages and use them locally for your own development and to git-version them separately. extract sets up local-dev mode that lets you work on the package in its source folder as if connected via source-directories but behaving like a normal dependency in your elm.json.

The 0.6-way of sharing for your scenario would be to keep your private packages in a parallel packages folder and then let each developer install them once into cache using wrap package cache --from-file PATH PACKAGE looping with a single script you keep in the same repo. (You’d set up a structure corresponding to ELM_HOME layout: packages/author/name/version, that way you can also version your packages nicely).

From there, the usual elm install my/private-package just works: if the package is in the cache, elm install is happy. wrap takes care not to confuse elm about which packages came from where, so your normal elm install will continue functioning as usual, picking up new public packages. And that also works in CI as long as you do the cache --from-file step before calling elm make.

The other way is to commit to git-repo-per-package and then every developer installs with wrap package install --from-url URL PACKAGE: this has the same effect as installing from the public repository. (The public repository just redirects elm install to Git to download the zipball from url and unpacks it in ELM_HOME. wrap install –-from-url does the same with explicit url of the zipball, without needing the initial metadata).


Upcoming version 0.7 brings local repository: you extract a package and then push it to local repository, similar to how you would publish to the public registry now, but without the tag-then-push-to-Github dance. (Behind the scenes it is really the ELM_HOME layout with more metadata and clean CLI UX to manage; no web servers/databases involved).

After that, my plan is to spend some time on policies: things that let you decide what you want in your repos and what you don’t want; (think of it like git pre-commit hooks). Then 0.9 brings us team/org package repositories.

Local private repository is easier and more robust to deliver than team/org as the latter require more setup (web servers, network rules, authentication, etc). But that doesn’t mean we can’t test things in advance if you are interested in solving the problem.


If you are interested, I’m curious about your environment. For example, when you say enterprisey, that usually rhymes with Windows. Are your servers also Windows? And would you/your org be open to beta test via a hosted private team repo? (That is, hosted on elm-wrap.cloud, with right authentication and security). Much easier during testing than also having to provide installation support.


One more thing: 0.6.1 doesn’t mean “not tested” or “in beta.” It just signals “not all features are in there yet,” like Elm versioning. All releases are production grade. (Betas are marked ~preview.{1,..,n}).

6 Likes

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.