How to replace a core package (elm/url) with a fork?

[domenkozar]

Hi all,

I need to use a fork of elm/url due to a bug that resulted into quite a few production issues.

What is the best way to provide my own fork of elm/url to elm.json?

Domen

[Herteby]

Url.Builder doesn’t contain any kernel code though, so you could publish a package containing just a fixed version of that module. (You’ll have to name it Url.BuilderFixed or something to avoid the name collision)

[domenkozar]

A bit more background: we are using openapi-generator to generate a lot of glue code to interact with the backend. openapi-generator glue uses elm/url to issue HTTP requests.

So ideally I’d be able to tell Elm that elm/url lives somewhere else now.

Url.BuilderFixed solution is tricky because:

1. other modules could use it that I have no control over like third party packages (can be avoided by auditing)

2. it’s easy to forget not to use fixed version (can be avoided by CI checks)

3. involves quite a bit patching (introduces indirection)

However all three are addressable, so this solution might work.

[hpate]

Some of the comments in this thread were removed, I am guessing due to them discussing kernel code restrictions.

Is even discussing it against the rules?

[andys8]

Is even discussing it against the rules?

Just … Wow. I listed 5 options in total.

Regarding openapi: openapi-generator/docs/customization.md at master · OpenAPITools/openapi-generator · GitHub could help. It’s possible to customize type mappings and a bunch more. Not sure if applicable for the issue.

[rupert]

@hpate

Is even discussing it against the rules?

Heretics.

[utkarshkukreti]

I have a similar problem with a core package (elm/bytes) and would like to know the best way to use a fork of elm/bytes. Here’s the issue that doesn’t received a response in 2 months. I’m using an alternative solution which is too slow for large data (1MB+) and uses 16x memory. (Should I open a new thread?)

[akoppela]

Also it’s possible to add post build hook (e.g. with gulp or whatever other tool) which will patch compiled JS code. I saw some people did that before.

[jerith]

Here is how I’m applying my fixes from elm/core #1018 while I wait for that PR to be resolved.

In words: I added a custom patch phase to the standard nix build process for elm projects, which applies the patch(es) from the issue after the base elm/* code is fetched into $ELM_HOME by elmPackages.fetchElmDeps.

[francescortiz]

What I would love to see is private repository registries were you can do whatever you want… I wish I had the time to do it

[rupert]

I liked the feature in elm-github-install that let you override a package with a new location. An example from its README:

"dependencies": {
    "gdotdesign/elm-install-test": "1.0.0 <= v < 2.0.0",
    "elm-lang/core": "5.0.0 <= v < 6.0.0",
    "elm-lang/svg": "2.0.0 <= v < 3.0.0",
    "elm-lang/dom": "1.1.1 <= v < 2.0.0"
  },
  "dependency-sources": {
    "elm-lang/core": "git@github.com:someuser/core",
    "elm-lang/dom": "../elm-lang/dom",
    "gdotdesign/elm-install-test": {
      "url": "gdotdesign@bitbucket.org:gdotdesign/elm-install-test",
      "ref": "master"
    }
  }

Though I think it would be better if the dependency-sources section that does the overrides was placed in a separate file than in elm.json.

I believe there are legitimate cases for wanting to fork even the official elm/ pacakges with native code. Its never a good idea to introduce something like that into your project for gratuitous reasons like wanting to avoid using a port when that is a perfectly good solution to your problem. One of the strongest reasons for wanting to do this would be so that a fork of a package can be worked on and tested prior to submitting a PR for a fix.

[hpate]

There was some discussion on 0.19 support for it here https://github.com/gdotdesign/elm-github-install/issues/62

[rupert]

A lot of the discussion there seems to centre around how to have it install native code from any module, regardless of it being in the elm or elm-explorations namespace. For the purposes of this discussion it would be enough to just allow an override for a forked version of one of these official packages to be installed, without breaking the native code restriction. Again, I should stress, not to try and jail break Elm native code restrictions, but purely to make it easier to develop and test forks in order to fix bugs.

[domenkozar]

That’s an interesting take. I’d need to see how that would work with Parcel, but maybe we should just symlink node_modules and Elm caches from Nix builds.

[domenkozar]

@rupert

Again, I should stress, not to try and jail break Elm native code restrictions, but purely to make it easier to develop and test forks in order to fix bugs.

That’s my intention as well. I don’t know if the fix I upstreamed works correctly for my use case.

Reading the discussion and thinking how to fix this, it seems like forking the compiler and implementing a workaround seems best at the moment for “private” use cases. If somebody has a better idea, please contact me at domen@hercules-ci.com or just send me a message here.

[evancz]

Given what I am reading, it sounds like the best path is to create a USER/url that depends on elm/url and reexports everything except for the functions you want to change.

As noted earlier, the function in question is a pure function, so you can the define it however you want.

I would like to fix this, but there are a great deal of projects going on right now that make it hard for core team folks to look into this right this second. Steps I’ll need to do are (1) verify with standards the behavior and (2) figure out if this is a MAJOR change. I also am trying to do 0.19.1 and unblock some work in elm/core. I understand that it’s frustrating that things that look easy can actually be very difficult in practice.

I hope that the solution I mention here (i.e. a USER/url package with re-exports) is the path you will take for now.

[domenkozar]

Hey Evan,

there’s absolutely no pressure from my side to address these. I’m a fan of focused work so I understand Elm 0.19.1 is in the pipeline.

I’d appreciate having an indirection for dependencies so they can be replaced with forks, to compensate for the time needed for upstream to review, test and merge.

Reexporting elm/url is an option, but it comes with a non-trivial maintenance cost. As our API is generated those files would need to be patched so they’d start using the new package.

In that case, I’d rather patch elm and provide an alternative binary for development and deployment rather than fiddle with the source everywhere (it also comes with cognitive overhead, being aware you can’t just import Url).

[evancz]

@domenkozar

Reexporting elm/url is an option, but it comes with a non-trivial maintenance cost.

What is the maintainance cost exactly?

After forking the repo and replacing the definitions with func = Url.func, I don’t see what maintainance cost there would be after that.

@domenkozar

As our API is generated those files would need to be patched so they’d start using the new package.

It sounds like you’d have to:

1. Change "elm/url" to "USER/url" in an elm.json file
2. Change import Url to import MyUrl as Url when generating these files

Based on my experiences writing packages and working on the compiler, I think one could do everything on the USER/url path faster than GHC can even build the dependencies for the Elm compiler. And you avoid needing to worry about distributing binaries to team members and CI and making sure there are no issues on different platforms.