State of conflicting modules with elm 0.19

Show and Tell
1

Following a discussion about namespaces from another topic, I was wondering about the current state of conflicting exposed modules in elm 0.19.

So to have a better picture of the conflicts currently present in the packages repository, I made a quick script and the results are here:

We get the following numbers:

  • 566 elm 0.19 packages found
  • 1491 unique exposed modules names found
  • 189 conflicting modules found (around 13% of unique exposed modules names)

Those are actually often forked packages, pre-releases or very similar packages so I cleaned it up a little with the following (quite arbitrary) rules to have a better picture of the potential real-world issues (as it would be quite unexpected to use several at once, except maybe for testing or when transitioning):

  • ignore conflicting module when all packages are from the same author
  • ignore conflicting module when packages have almost the same name
  • ignore Bootstrap or Bulma packages

I get the following list:

{
  "Base64": [
    "danfishgold/base64-bytes@1.0.1",
    "truqu/elm-base64@2.0.4",
    "waratuman/elm-coder@3.0.0"
  ],
  "Color": [
    "avh4/elm-color@1.0.0",
    "elm-athlete/athlete@4.0.0",
    "tesk9/palette@1.2.0",
    "the-sett/elm-color@1.0.1"
  ],
  "Color.Convert": [
    "elm-athlete/athlete@4.0.0",
    "noahzgordon/elm-color-extra@1.0.1"
  ],
  "Color.Interpolate": [
    "elm-athlete/athlete@4.0.0",
    "noahzgordon/elm-color-extra@1.0.1"
  ],
  "Color.Manipulate": [
    "elm-athlete/athlete@4.0.0",
    "noahzgordon/elm-color-extra@1.0.1"
  ],
  "Csv": [
    "jonoabroad/commatosed@1.0.0",
    "lovasoa/elm-csv@1.1.5",
    "zgohr/elm-csv@1.0.1"
  ],
  "DatePicker": [
    "CurrySoftware/elm-datepicker@3.0.1",
    "abradley2/elm-datepicker@4.0.1",
    "goilluminate/elm-fancy-daterangepicker@3.2.0"
  ],
  "Diff": [
    "indicatrix/elm-input-extra@1.0.1",
    "jinjor/elm-diff@1.0.5"
  ],
  "Dropdown": [
    "fedragon/elm-typed-dropdown@2.0.2",
    "hercules-ci/elm-dropdown@1.0.1",
    "indicatrix/elm-input-extra@1.0.1",
    "indicatrix/elm-input-extra@1.0.1"
  ],
  "Force": [
    "gampleman/elm-visualization@2.0.0",
    "ianmackenzie/elm-units@2.0.2"
  ],
  "Form.Input": [
    "bluedogtraining/bdt-elm@26.0.1",
    "etaque/elm-form@4.0.0"
  ],
  "Grid": [
    "bluedogtraining/bdt-elm@26.0.1",
    "joakin/elm-grid@1.0.1",
    "the-sett/the-sett-laf@6.0.0"
  ],
  "Matrix": [
    "Kraxorax/elm-matrix-a@2.0.0",
    "ryry0/elm-numeric@1.0.0"
  ],
  "OrderedDict": [
    "wittjosiah/elm-ordered-dict@1.0.1",
    "y0hy0h/ordered-containers@1.0.0"
  ],
  "Plot": [
    "Bractlet/elm-plot@1.0.1",
    "NoRedInk/elm-plot-19@1.0.0",
    "terezka/elm-charts@1.0.0"
  ],
  "Rating": [
    "Bernardoow/elm-rating-component@2.0.2",
    "FordLabs/elm-star-rating@1.0.0"
  ],
  "Tree": [
    "miyamoen/tree-with-zipper@1.0.0",
    "zwilias/elm-rosetree@1.3.1"
  ],
  "Tree.Zipper": [
    "miyamoen/tree-with-zipper@1.0.0",
    "zwilias/elm-rosetree@1.3.1"
  ],
  "Validate": [
    "kirchner/form-validation@1.1.1",
    "rtfeldman/elm-validate@4.0.1"
  ],
  "Validation": [
    "ericgj/elm-validation@2.0.0",
    "iodevs/elm-validate@3.0.3",
    "ozmat/elm-validation@2.2.1"
  ]
}

I don’t have myself a conclusion yet, feel free to comment.

Here is the full modules list if you want to process it:

PS: these lists are only about packages exposed modules. I’m not sure if packages internal modules can also conflict, does anyone know?

7 Likes
2

I would like to see what the list looks like without module prefixes. I suspect there are plenty of us that have prefixed modules with essentially the package name to avoid conflicts.

3

A very helpful survey, thanks.

Internal modules cannot conflict, I am pretty sure.

1 Like
4

A few suggestions that could start a conversation around this:

  • The elm runtime could check for these kind of conflicts when an author is first publishing their package and warn them that something similar already exists. What to do about that would be something to discuss, but a few things that come to mind:
    • Warning only, link to the current conflicts
    • Not allow similar packages to be published without some explicit reason as to why this package is different
    • Foster collaboration somehow to merge useful functions from both packages if that makes sense
  • Set up work groups with package authors who have conflicts to make the package system more robust - allow them to identify the best portions of each implementation and create a super package.
  • In general, allow removal of packages from packages.elm-lang.org. If this is possible already, it’s not advertised enough.
  • Contact package authors when maintenance is required. If no response is obtained, have a mechanism to take over a package rather than forking and publishing a new one.

Probably more things that could be done, but this was just a stream of consciousness dump.

5

@gampleman, here you go:

I considered a prefix a common string before a first . in the module name that is shared by all the exposed modules of a package.

We would get the following numbers if prefixes were removed:

  • 566 elm 0.19 packages found
  • 1417 unique exposed modules names found
  • 207 conflicting modules found (so this would raise conflicts to around 15% instead of 13%)

I cached the full modules names list if you want to play with it, see my edited first post.

6

I have made an Observable notebook with some of these analyses which is easy to fork and modify for further analysis:

4 Likes
7

Because conflicts do not break anything in the packages repository, I believe that if there is the will to solve them once for all, solving them at the compilation stage could be more effective and would lead to a simpler modules naming process globally (no publish checks, prefixes or namespaces needed).

One way that comes to mind to solve it would be to allow to prefix packages during installation

For example, as:

$ elm install ianmackenzie/elm-units

and

import Force

leads to a conflict with gampleman/elm-visualization Force module, an optional supported syntax that would be hinted by the compiler on conflicts could be:

$ elm install ianmackenzie/elm-units as Units

which would store the Units prefix in elm.json.

Then the elm-units modules would be imported and used with the Units prefix, for example:

import Units.Force [as ...]

This way, packages developers would not need to worry anymore about conflicts as they would know that users can solve them, so they would use simple modules naming, without prefixes or avoiding common names.

Some nice properties of this solution are that:

  • it would be backward-compatible with current modules
  • it is quite intuitive as it is close to the import syntax
  • users who do not have conflicts can ignore it completely
9 Likes
8

Or even without involving elm-install one could simply do:

 import Force from ianmackenzie/elm-units as Units.Force
 import Foce from gampleman/elm-visualization as Viz.Force

This way the disambiguation could only be done in modules that actually require ambiguous modules.

9 Likes
9

I do like the import Some.Module from author/package as New.Name approach as an optional way to disambiguate. I’ve added the Force module name conflict as a new issue referencing #1625…in the meantime, I’m happy to discuss workarounds if needed!

I’m not too keen to make breaking changes to elm-units at this point, but one potential solution would be to publish a new elm-units-prefixed package with modules like

module Units.Force exposing (Force, Newtons, newtons, inNewtons)

import Force

type alias Force =
    Force.Force

type alias Newtons =
    Force.Newtons

newtons : Float -> Force
newtons =
    Force.newtons

inNewtons : Force -> Float
inNewtons =
    Force.inNewtons

That is, elm-units-prefixed would depend on elm-units and basically re-export its types and functions under prefixed module names. I think that would let you avoid the module name conflict, while still being able to interoperate with other packages that used elm-units directly (since the types themselves would be the same). It would avoid having to make any changes to elm-units itself, and it would make it fairly clear that elm-units-prefixed is a (hopefully) temporary workaround that you should use only if necessary. And with a bit of work the prefixed package could probably be auto-generated with a script =)

10

I would propose package authors to prefix with a vendor name as well. For example IanMacKenzie.Units.Force in this case (use your github name). That is how many other languages avoids these kinds of problems.

I would not worry to much about changing module names or prefixing them. Yes, it is technically a breaking change, but it is very easy to fix over the whole code base for the consumers of your library.

If the modules are imported in to many places to deal with it manually, something like this:

grep 'import Force' -lr src/ | xargs sed -i 's/import Force/import Unit.Force as Force/g'

will work for 95% of the cases.

Also, most IDE:s (if someone uses that) has some refactor magic tool to help with this kind of things.

Your approach would only work if all types are opaque (don’t know if they all are) and no constructors are exposed.

Elm is very clear on what is considered a breaking change. However, in practice I think it is much more nuanced. A breaking change isn’t a problem if it is easy to fix and can be automated.
Anyway, this is just my opinion.

11

Prefixing is certainly one way to avoid conflicts, but I’ve found that having the default being un-prefixed has some interesting effects on package design. For example, I used to have an opensolid/geometry package that had module names OpenSolid.Point3d etc. which were designed to avoid conflicts. In the spirit of literal names, that package is now ianmackenzie/elm-geometry with modules such as just Point3d. Where before I could be a bit lazy with API design since I could just reason that “well this is my Point3d type, you’re welcome to also use your own”, that gets harder if the module name is just Point3d. It’s forced me to be a bit more careful, and try to design a more universal API where I can say “I think this should be the Point3d module to use”.

In general I think there’s a difference between “what approach solves the immediate problem” and “what approach nudges the ecosystem in a good direction”. If one of the goals of Elm is to have a smaller number of higher-quality packages, I think unprefixed-by-default modules is arguably a good approach (although I do think there should probably be an escape hatch like import ... from ...).

I think that is true for packages that are used only by applications, but one of my goals for elm-units specifically is that it can be depended on by other packages. Then if you make breaking changes you can get into ugly situations like

  • I update elm-units from 2.0 to 3.0
  • a new author1/package1 is published and depends on elm-units 3.0
  • but there’s an existing author2/package2 that still depends on elm-units 2.0

Then nobody can include both author1/package1 and author2/package2 in an app, since there’s an unresolvable dependency conflict. This has already come up a few times with elm/http, since some packages depend on version 2 and some other packages still depend on version 1.

Excellent point! In the case of elm-units that can be worked around, but you’re right that the approach wouldn’t work in all cases.

5 Likes
12

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.