Some ideas for the package directory

#1

Lots of packages do foundational work towards solving a problem and designing a good API. But often there’s more to be done! The way the package directory is structured, community contributions are isolated from one another. This means:

  • Package maintainers may have a hard time finding and commissioning someone with domain knowledge to extend their API
  • Folks with domain knowledge may have a hard time contacting and coordinating with package authors, PR merges can be delayed a long time, etc.

Take gampleman/elm-visualization for example. Say you are a specialist in statistics and so, for your graphs, you implemented a trend fitter for Paths. The community might really like that functionality! So you make a package and in the package description, you write

Generate trendline equations for elm-visualization's Paths

Sure, that works. But your description now has to do two jobs: specify what problem your package solves, and what context it works in. Also, people who use elm-visualization may have no idea that it now has this functionality. They have to either stumble across your package, or @gampleman will notice it and kindly update the README to advertise your package. And if he disapproves of your project design for whatever reason (goes against the design philosophy, leads to antipatterns, etc.), he can’t directly do anything about it, he’ll just have to kindly ask you to change or remove your package. Maybe a user began their project with a different solution, like gicentre/elm-vega, but need your functionality, so they have to incorporate competing solutions for the same thing in their codebase.

For instance, we see this pattern in the description of ianmackenzie/elm-geometry-linear-algebra-interop

Interop support for ianmackenzie/elm-geometry and elm-community/linear-algebra

@ianmackenzie has the advantage of being the author of one of the packages. But I’m concerned with how this system scales. With all packages being completely independent of one another in the package directory, in terms of the package directory application, I feel this community structure inhibits the best kind of contributions: contributions that build off other contributions!

This is why I think a way for the package directory to help authors manage the ecosystem surrounding a package would be helpful for the elm community.

This comes with a number of benefits:

  • Users get more functionality, of course. I want to see a mdgriffith/elm-markup markup parser default for mdgriffith/elm-ui! It just so happens these two packages are written by the same person, but they have really good synergy! And I imagine that as Elm’s tooling continues to mature, the relationships between packages will get increasingly complex. Lots of other, otherwise disparate packages might turn out to be a great combo, such as perhaps BrianHicks/elm-trend and terezka/line-charts (The former, by the way, mentions in its readme that it wants help! It would be great to have a standardized way Brian could define to extend his package according to his API design, in the case that someone comes along who graciously wants to contribute their domain knowledge).

  • Encourages the community to build itself around a unified, foundational tool rather than hop around competing solutions. Searching “forms” in the package search gives you six competing API designs! How is a user supposed to compare all of these at once? More than likely they’ll look at maybe two or three unless they are really determined to have the best forms ever. There’s nothing wrong with new solutions, and I can’t speak for the packages here, but hopefully you see the point that those unsatisfied with the features available in one solution could contribute to rather than compete with it.

  • Encourages package authors to follow the productive pattern of having easy to use, opaque, front-facing modules and highly parameterized, transparent, extensible modules. By giving a clear specification, contributors to the package know exactly how the API is supposed to be extended. I think hecrj/composable-form does this really well! This would be prime example of an opportunity to see the community’s complex form implementations! Perhaps Elm’s dependency manager could allow a front-facing option for apps and an extensible option for package extenders. Encouraging authors to develop extensible modules for their packages would also be a really cool way to find finely specialized or compositional datastructures, like a multigraph of bloom-filter sets of R-trees of linked Fibonacci heaps of skip lists of double-ended queues of matrices.

I don’t have a particular design/implementation in mind, just wondering what folks think of the idea, if it would be a worthy contribution down the road, how far down the road, etc.

Oh, and it might also be nice to have tags, like StackOverflow’s tag system where authors are discouraged from creating original tags. Other ideas about package search are discussed here.

What do you think?

Cheers,
Thomas Kagan

4 Likes
#2

Great post, I fully agree that package.elm-lang.org should be more “explorable” and make it easy to find related packages and packages adding additional functionality. A simple way to add basic support for this would be to display Dependencies and Dependents, like npm does it (but maybe not exactly like npm does it, since it doesn’t give you a whole lot of information.

What I personally would also like to be able to do is specify weak or optional dependencies for packages. For example, a package author may want to provide a library for some component, but might also want to add elm-ui support, or support for animations. Since we already have dead code elimintation in 0.19, they can already do this without blowing up the binary for folks not using those features. However, people might get discouraged from using packages that depend on ${some big library that they don't use}, even though this dependency is completely optional. It would also bring the ecosystem closer together by eliminating the need of a lot of *-interop packages.

Because of the compiler, we can generally aim at bigger/more complete packages than you could in Javascript, which seems to focus a lot on “single-function packages”. It also encourages colaborating more, since new functionally can always be added --for free-- (not really) to existing packages. This is also encouraged by the current naming strategy of “don’t be creative with your package name”

1 Like
#3

Great ideas, it would be nice to allow packages to have optional dependences, eg for Graphs with an optional Matrix dependency

#4

It seems like some folks like the ideas I laid out in my OP. Fantastic!

I was wondering, for perhaps me, but more so any highly motivated visitors of this page: if one were to formally propose these features, how would they do it? Who does this saintly individual need to talk to to actually get this done?

closed #5

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.