"PROBLEM BUILDING DEPENDENCIES" error in pipeline (SOLVED)

[lucamug]

TLDR; If you are getting some random error from the Elm compiler like the one below, make sure that you are not running two compilations processes at the same time on the same machine for two different projects, as this could be an issue.

---

Let me move from the Slack chat to here the thread related to an issue (and a resolution), so that it became searchable in case other people face something similar in the future.

At work, we had issues with this type of error during a pipeline of a monorepo hosting several Elm projects:

-- PROBLEM BUILDING DEPENDENCIES -----------------------------

I ran into a compilation error when trying to build the following package:

    fredcy/elm-parseint 2.0.1

This probably means it has package constraints that are too wide. It may be
possible to tweak your elm.json to avoid the root problem as a stopgap. Head
over to https://elm-lang.org/community to get help figuring out how to take this
path!

Note: To help with the root problem, please report this to the package author
along with the following information:

    elm/core 1.0.5

If you want to help out even more, try building the package locally. That should
give you much more specific information about why this package is failing to
build, which will in turn make it easier for the package author to fix it!

The weird thing is that these errors were happening on a random base and they were referencing random packages.

It seems the issue was about a conflict between the Elm caching system and the Nx/Jenkins caching system.

Nx (https://nx.dev/) and Jenkins (https://www.jenkins.io/) are the technologies used in our pipeline and they support the parallel execution of processes.

The Elm compiler caching system works on two separate layers.

One is located in the folder “elm-stuff” inside the project folder. The other is located in the folder “.elm” under the $HOME folder (or $ELM_HOME if defined).

While each Elm project has its own elm-stuff folder, .elm is shared among all the Elm projects on the same machine.

This caused issues when multiple instances of the compiler were running at the same time on the same machine, in parallel.

A resolution was to create an independent .elm folder for each project with

export ELM_HOME=${HOME}/.elm_${PROJECT_NAME}

This has the consequence that the same package is downloaded more than once if used in multiple projects, but this only happens the first time a project is created. After that, the Nx cache should kick in, and only newly added packages should be downloaded.

Any better suggestion is welcome.

I heard that similar issues are happening randomly on Netlify. Not sure what that could be, but it may still be related to some caching conflict. I don’t know if this issue could happen also if several compiler processes are happening at the same time but they belong to the same project, instead of different projects (the monorepo case).

Prior art and references

Skinney/murmur3 not downloading

I am seeing the following stack trace in my ci/cd pipeline when installing elm packages. 2:08:33 PM: Starting downloads... 2:08:33 PM: ✗ Skinney/murmur3 2.0.8 2:08:33 PM: ● elm/random 1.0.0 2:08:33 PM: ● elm/regex 1.0.0 2:08:33 PM: ● elm/time 1.0.0 2:08:33 PM: ● elm/html 1.0.0 2:08:33 PM: ● elm/parser 1.1.0 2:08:33 PM: ● truqu/elm-base64 2.0.4 2:08:33 PM: ● elm/browser 1.0.2 2:08:33 PM: ● elm/virtual-dom 1.0.2 2:08:33 PM: ● elm/bytes 1.0.8 2:08:33 PM: ● elm/url 1.0.0 2:08:…

$NETLIFY_CACHE_DIR no longer defined?

Caching Elm downloads on Netlify builds

Hello all, Evan requested that people take some time to make sure their Elm downloads are being cached on their CI environments. I came up with a solution for Netlify builds. Here’s what the change looks like in your netlify.toml config: [build] publish = "./dist/" - command = "npm run build" + command = "export ELM_HOME=\"$NETLIFY_CACHE_DIR/elm\" && npm run build" I updated the elm-pages-starter repo with that config for anybody who uses that template going forward: The details Here’…

Caching Elm Downloads on Netlify Builds (Reprise)

All, There have recently been changes to the Netlify build system, and Dillon’s recommendation for caching Elm packages no longer works. They now have a generic caching npm package that you can use. For example, this is how the netlify.toml file looks in my projects: [build] publish = "dist" command = "export ELM_HOME=\"elm-stuff\" && npm run build" [[plugins]] package = "netlify-plugin-cache" [plugins.inputs] paths = ["elm-stuff"]

"PROBLEM BUILDING DEPENDENCIES" (solved)

I ran into a vague build problem recently, and I’m posting it because I didn’t find other cases of it when googling. When building with netlify, but not locally, I would get this error from the elm compiler: 3:41:29 PM: > elm-pages build 3:41:33 PM: -- PROBLEM BUILDING DEPENDENCIES ----------------------------------------------- 3:41:33 PM: I ran into a compilation error when trying to build the following package: 3:41:33 PM: ianmackenzie/elm-units 2.9.0 3:41:33 PM: This probably means it …

[dillonkearns]

Thank you so much for investigating this and finding the cause Luca!

@lucamug

I heard that similar issues are happening randomly on Netlify.

I think it’s likely that this is because a lot of elm-pages users use Netlify, but not actually related to Netlify. The elm-pages build command is running two parallel compilation tasks for the client and worker apps:

dillonkearns/elm-pages/blob/f7248851e604b2bdf91b65162f0cad5a6c637f55/generator/src/build.js#L71-L73

In the upcoming release this is no longer happening, so that race condition should be avoided. Good to know what was the source of it finally, thanks again for sharing your findings!