Reusing a non-published module across multiple apps

christian 2017-12-11 23:35:59 UTC #1

At the beginning of 2017 we started using Elm in production at my company, and I am happy to say we now have 7 Elm applications totaling over 35,000 lines of Elm code. Our team is fairly small but I can already see a problem looming on the horizon: organization and re-use of non-published modules.

Each of the seven different Elm apps in our software are loaded on various pages of a monolithic MVC application. With this in mind, there are certain modules that make sense to reuse. However, for a variety of reasons I would not consider them open-source quality and do not think they are good candidates for publishing on package.elm-lang.org.

Currently our Elm project directory setup is something like this.

app/
    ...non-Elm-related files...
    webroot/
        /elm-dist -- symlink to /elm-app/dist
elm-app/
    dist/
        first-elm-app.js
        second-elm-app.js
        third-elm-app.js
    first-elm-app/
        /src
            BootstrapHelper.elm
            ...other files...
        elm-package.json
        build-script
    second-elm-app/
        /src
            BootstrapHelper.elm
            ...other files...
        elm-package.json
        build-script
    third-elm-app/
        /src
            BootstrapHelper.elm
            ...other files...
        elm-package.json
        build-script

In the example above, BootstrapHelper.elm has to be copy-and-pasted to each Elm app’s /src directory. In each case, the file is essentially identical, but may receive small incremental changes. There are several problems I see with this approach.

The initial copy-and-paste takes some work (especially when you aren’t sure which version is the “best” one to copy)
There is a possible divergence in the API/types of of the module as it is expanded to accommodate the needs of the various apps

I have seen some npm packages to help manage this problem, but that was quite a while ago. I would appreciate any feedback from anyone who has been in a similar situation and has a solution they would recommend to solve the scenario described above.

EDIT: I realize that there may be a good solution to this problem in Elm 0.19. If that is the case then I am happy to wait.

joakin 2017-12-12 00:20:08 UTC #2

It is very cool to see usage in your company!

I’m not sure if this could help you somehow but maybe you can leverage source-directories in elm-package.json, see https://guide.elm-lang.org/reuse/modules.html :

{
    ...
    "source-directories": [
        "src",
        "benchmarks/src"
    ],
    ...
}

Other links:

Maybe join the sources in the same folder so that they can be referenced? Something like:

elm-app/
    elm-package.json
    src/
        BootstrapHelper.elm
        first-elm-app/
            ...
        second-elm-app/
            ...
        third-elm-app/
            ...
    build/
        build-first-elm-app
        build-second-elm-app
        build-third-elm-app
    dist/
       ...

You’d also keep one elm-package.json so that your apps use the same version of dependencies more easily.

Just some thoughts

avh4 2017-12-12 01:04:49 UTC #3

At NoRedInk, we tend to just publish packages even if they aren’t well-documented or don’t make sense for anyone else to need

For my personal projects, I use git submodule to check out the shared code into the projects that need it and add a source-directories entry to package.json as joakin mentioned. This can be painful because you’ll also have to add any package dependencies of the shared code to each of the projects’ elm-package.json. (git subtree would work equally well if you prefer it over git submodule.)

christian 2017-12-12 05:08:33 UTC #4

@joakin Thanks for the ideas! I just tried adding a sibling to all my existing apps called Shared and created an src-shared symlink to it in each of the other projects. So my setup is now…

    elm-app/
        dist/
            ...
        first-elm-app/
            /src
                ...
            /src-shared -- symlink to ../Shared/src
            elm-package.json
            build-script
        second-elm-app/
            /src
                ...
            /src-shared -- symlink to ../Shared/src
            elm-package.json
            build-script
        -- other elm apps...
        Shared/
            /src
                /Shared
                    BootstrapHelper.elm
                    -- other shared modules...
                Shared.elm
            elm-package.json
            build-script

And every elm-package.json contains ["src", "src-shared"] as the source-directories array. This allows me to import shared modules like the one above as Shared.BootstrapHelper in any of the other Elm apps.

Another neat feature of this setup is that my Shared.elm module is essentially just a Hello World app that imports all of my shared modules, so running the build script for it will ensure there are no errors in any of the submodules. I am likely going to add a build-all script that calls all the other builds that can be used to rebuild everything and ensure there are no breaking changes.

@avh4 Thanks. That’s what I figured some do. I suppose that there are benefits to doing so that come with publishing packages (e.g. the semver constraints that ensure you aren’t breaking existing APIs). I will consider this in the future.

rupert 2017-12-13 10:29:02 UTC #5

The way I do it is to put the shared code in its own package, exactly as if it was going to be published, but keep it in my local git repo. Then use elm-github-install to reference it:

"dependencies": {
    "rupertlssmith/elm-auth": "1.0.0 <= v < 2.0.0",
    "elm-lang/core": "5.0.0 <= v < 6.0.0",
    ...
},
"dependency-sources": {
    "rupertlssmith/elm-auth": {
        "url": "rupert@gitlab:rupert/elm-auth",
        "ref": "master"
    }
}

Allows me to share the client side authentication code across all my projects, without copying and pasting it or using git submodules, or publishing it.

zkessin 2017-12-28 09:15:35 UTC #6

I think it would be good to have as a future option the ability to have a private elm package repo for a company. So we could have elm-package.company.com with all the various internal packages. This would have advantages over just using a git repo in that as a company scales up the use of Elm it would give them internally all the same guarantees that the public elm package repo has (SemVer, Visibility etc).

I assume that at some point we may have enterprise companies working with Elm who might build a large number of internal pacakges.