Exposing a subset of custom type constructors in 0.19

[jwoLondon]

I notice (an undocumented) change in elm 0.19 is to disallow the exposure of a subset of custom type constructors in a module.

I hope I have misunderstood this, or there is a workaround, but on the face of it this seems like a real killer to me as I make very extensive use of partially exposed custom types. I did this, following advice on this board, in order to avoid major version bumps if I needed to add new constructors to an existing custom type (I simply hide the new constructor and create an exposed function that generates the type with that hidden constructor).

Is there a way round this to partially expose type constructors without making a major API change? (I could hide all existing constructors and add a new function for each, but this would be a very major change to the package API and is likely to annoy users of my published modules).

What is the rationale for this restriction?

I love most of the changes in 0.19 and appreciate the effort that has gone into this over the last couple of years, but this particular change has really caught me by surprise (were there any indicators this was coming?) and risks causing major problems if I can’t find a workaround that doesn’t impact the package API to such a major extent.

[folkertdev]

One of the reasons is that we primarily use constructors for pattern matching, and pattern matches need to be exhaustive. When only a subset of the constructors is exposed, one can’t pattern match on that type.

When pattern matching is not something you need on a type, you should probably wrap the constructors in functions. This has the additional benefit that each case can be clearly documented.

I also don’t really see the benefit in your case: if you’re going to wrap the new constructors anyway, wouldn’t it be more consistent to wrap them all? Also I think that, with the help of the compiler, this won’t be that big of a change in practice for your users. It’s a simple search/replace all.

[bChiquet]

This change is also a heartbreaker for me, as being able to explicitly name what was imported made it much easier to know where things came from. That’s why, for instance, import * has bad rep in Python.

Also, when a type is subject to evolution, this is a good way to futureproof code having pattern matches:

module A exposing (A (Some, Many, ALOT))
type A = FutureProof | Some | Many | ALOT

module B exposing (..)
import A (A(Some, Many))

function : A -> Result String Int
function a = case a of 
   Some -> Ok 1
   Many -> Ok 3
   _    -> Err "Not handled"

[jwoLondon]

@folkertdev

One of the reasons is that we primarily use constructors for pattern matching…

I agree this is one of the reasons, but not the only one. Another is to provide self-documenting custom types that are convenient for people to use. As @bChiquet says, the new change prevents them being used for that purpose for types subject to evolution (at least not without breaking changes if a new constructor is required).

@folkertdev

When pattern matching is not something you need on a type, you should probably wrap the constructors in functions. This has the additional benefit that each case can be clearly documented.

I also don’t really see the benefit in your case: if you’re going to wrap the new constructors anyway, wouldn’t it be more consistent to wrap them all?

An example in my case, I need to represent different cursor types but with the possibility that in the future more cursor types might be added. Currently I have this:

type Cursor
    = CAuto
    | CDefault
    | CNone
    | CContextMenu
    | CHelp
    | CPointer
    | CProgress
    | CWait
    | CCell
    | CCrosshair
    | CText
    | CVerticalText
    | CAlias
    | CCopy
    | CMove
    | CNoDrop
    | CNotAllowed
    | CAllScroll
    | CColResize
    | CRowResize
    | CNResize
    | CEResize
    | CSResize
    | CWResize
    | CNEResize
    | CNWResize
    | CSEResize
    | CSWResize
    | CEWResize
    | CNSResize
    | CNESWResize
    | CNWSEResize
    | CZoomIn
    | CZoomOut
    | CGrab
    | CGrabbing

I am now going to have to create a 37 new parameterless functions and document each one (at least minimally). And for my package as a whole, it will be several hundred more. That is an additional problem because my API documentation is already at the 512k size limit, so something else will have to give to make way for it.

@folkertdev

with the help of the compiler, this won’t be that big of a change in practice for your users. It’s a simple search/replace all.

None of the changes for users are individually big, but it still means that existing codebases will break when people update the library, which, as Evan has said, is something API developers should avoid forcing on users of packages if at all possible.

Overall, I like Evan’s philosophy of “do it right, not do it now” with plenty of time for reflection on possible changes to the language between release cycles. And generally, I am really impressed with the 0.19 changes. But I feel this needs to work both ways - unannounced changes like this particular one seem be having exactly the opposite effect in my case at least.

[rupert]

@jwoLondon

An example in my case, I need to represent different cursor types but with the possibility that in the future more cursor types might be added.

Why not just expose the full Cursor type, and in the future when you add more, client code may break and have to deal with that change?

[bChiquet]

Why not just expose the full Cursor type, and in the future when you add more, client code may break and have to deal with that change?

In my case, because changes to some parts of applications may happen often enough, or in an automated fashion, that we did not want humans to react to change whenever it happened, but whenever it was needed.

If someone has an application that works for them but does not need 100% of the improvements you push, forcing them to adapt for 100% of these improvements feels pushy.

Following my previous example:
Package A maintainer: “I added ALOT, that lets you handle LOTS of things”
Package B maintainer, not interested.
Package A maintainer: “I improved the perfs for function foo”
Package B maintainer decides to upgrade and has to handle unrelated changes.

Hiding one constructor is a way to prevent this.

[mgold]

I’ll quote because I can’t say it any better myself:

@folkertdev

One of the reasons is that we primarily use constructors for pattern matching, and pattern matches need to be exhaustive. When only a subset of the constructors is exposed, one can’t pattern match on that type.

When pattern matching is not something you need on a type, you should probably wrap the constructors in functions. This has the additional benefit that each case can be clearly documented.

I think this is the solution you want, but @folkertdev didn’t spell out that you don’t need the many variants of the custom type. Hopefully an example will help.

module Cursor exposing (Cursor, auto, default, toStyle)

type Cursor = Cursor String

auto : Cursor
auto = Cursor "auto"

default : Cursor
default = Cursor "default"

toStyle : Cursor -> Html.Attributes.Attribute msg
toStyle (Cursor value) =
  Html.Attributes.style "cursor" value

This exposes the Cursor type but not the Cursor variant (formerly tag, constructor, etc.) so a consumer of this module can’t create arbitrary values of type Cursor. They can only use the two that you’ve exposed. And the only thing they can do with a value of type Cursor is call toStyle on it. You can now add as many other cursors as you like and it will only be a MINOR change.

@bChiquet

This change is also a heartbreaker for me, as being able to explicitly name what was imported made it much easier to know where things came from. That’s why, for instance, import * has bad rep in Python.

Don’t confuse imports and exports. I think you can still list all the variants of the custom types that you import, so string matching will find them and tell you what module they came from. However, the module B exposing (..) syntax is no longer valid, because you need to be more specific about your module’s interface.

[jwoLondon]

Thanks @mgold. That’s a nice compact pattern which I may adopt in places, but I doesn’t address my concern that the change forces a proliferation of exposed functions where previously there was just a list of (partially) exposed type constructors.

Internally, I think an opaque custom type for those with many constructors might be preferable as it is a little easier browsing the source code to see the type’s coverage.

[mgold]

@jwoLondon

a proliferation of exposed functions where previously there was just a list of (partially) exposed type constructors

Have you looked at the Html module? That’s certainly a lot of exposed functions.

A list of type constructors sounds minimal, but it’s not. There’s no way to go in and individually document each one. You have to awkwardly prefix each one with C and they all blur together.

How is the type consumed? You have to have some giant case somewhere, and you’re using the _ -> pattern because the type isn’t fully exposed, the compiler won’t tell you where that case is. In the pattern I showed you, you don’t have to update toStyle when you add a new Cursor value.

Point being, there’s a perfectly good (arguably better) pattern out there, and the language designers are trying to get you to use it.

[jwoLondon]

@mgold

Point being, there’s a perfectly good (arguably better) pattern out there, and the language designers are trying to get you to use it.

Fair point. This was what I was striving for in my original question: what is the rationale for the change.

I take the point that on balance the kind of pattern you mention can be preferable in many situations, but I think the advantages/disadvantages are nuanced and depend on context and other constraints.

I am not sure the ‘awkward prefix’ problem is solved though - the prefix is to minimise namespace clashes, but this would equally apply to the exposed functions that map onto each of the variants.

[bChiquet]

Hello @mgold,

Don’t confuse imports and exports.

The change effects both, I did the test on imports when I noticed the problem on exports.

When only a subset of the constructors is exposed, one can’t pattern match on that type.

As exposed in the example above, the _ -> [...] lets you pattern match on incompletely exposed constructors, and one of the interests for purposefully hiding a constructor is to force this pattern.

The aim is to communicate intent for custom types that evolve quickly, and to make sure that your users are always able to use newer versions. While exposing functions achieves this aim, it ruins pattern matching.

The offered workaround also imposes non-negligible quantities of boilerplate code, as well as more surface for mistakes (a constructor can’t do smart things…).

Point being, there’s a perfectly good (arguably better) pattern out there, and the language designers are trying to get you to use it.

Please consider that fair points were made in this thread, and that to this day they have received no answer.

[mgold]

@jwoLondon

This was what I was striving for in my original question: what is the rationale for the change.

I think that’s answered pretty well in the first response: to avoid needing _ -> to have an exhaustive pattern match. That pattern is considered less than ideal because if you add another variant to the custom type (to use the new terminology), the compiler won’t help you find all the places you may need to change.

@jwoLondon

I am not sure the ‘awkward prefix’ problem is solved though - the prefix is to minimise namespace clashes, but this would equally apply to the exposed functions that map onto each of the variants.

The Cursor module name is itself the prefix, e.g. Cursor.default. If you need to handle other things, give them their own module, so you can have Display.inline for example. In fact, I don’t think you even need toStyle; simply expose a bunch of values of type Attribute msg.

… Unless there’s something else you need to do with a Cursor, besides using it as an HTML attribute, in which case would you expand on that?

@jwoLondon

I am now going to have to create a 37 new parameterless functions and document each one (at least minimally). And for my package as a whole, it will be several hundred more.

Think of documentation as an opportunity, not a chore. But empty doc strings are accepted (I think), for exactly this sort of use. You also lose the giant pattern match that I assume you are doing to turn these variants into strings.

@jwoLondon

my API documentation is already at the 512k size limit, so something else will have to give to make way for it.

That’s a pity. Maybe split it up into multiple packages? Not ideal, I know…

@jwoLondon

the new change prevents them being used for that purpose for types subject to evolution

@bChiquet

The aim is to communicate intent for custom types that evolve quickly, and to make sure that your users are always able to use newer versions.

I’m not sure I understand this rationale. Arguably, if you are still figuring out the shape of your types, you’re not ready to publish a package. But otherwise, you’ll want to expose less of your type and control the interface very tightly. Instead of letting people at some of the tags, maybe provide a function that hands off Just the value in one of the variants you need to expose? I’m kind of flying blind here, since there hasn’t been any concrete examples given except for Cursor, and that’s based on a stable web API.

@bChiquet

Don’t confuse imports and exports.

The change effects both, I did the test on imports when I noticed the problem on exports.

Sorry, you are correct – import Maybe exposing (Maybe(Just, Nothing)) does not work even though it would import all the variants. So, maybe this is one more reason to keep your custom types opaque?

@bChiquet

The offered workaround also imposes non-negligible quantities of boilerplate code

Elm has boilerplate. Personally, I’ll take boilerplate over magic any day. To paraphrase Richard from Make Data Structures, it’s much better to write a few lines of easy-to-understand code if it prevents bugs – or gives a better API.

@bChiquet

as well as more surface for mistakes (a constructor can’t do smart things…)

A one-liner like default = toStyle "default" has less room for error than a giant case expression. You have to have the complexity somewhere. All the constructor does is add another named layer between the value and the behavior.

[bChiquet]

I didn’t say the code that needs to evolve was “still being figured out”. Some industries have constantly evolving code, simply because it is their normal state of business to be evolving and expanding their business. Thus, you can’t expect “mature” datastructures, but ever expanding cases.

Some products in these businesses, however, care for a subset of these structures only. It is normal for these to want to be out of the loop on new constructors, because adding a new is an active choice, and new items shouldn’t be added by default. The preferred workflow is opt-in rather than opt-out.

I have met this behaviour businesses who expand products in thin, vertical slices, rather than in a holistic way.

Personally, I’ll take boilerplate over magic any day.

If you allow me to reframe this in the context of our discussion, constructor = Constructor is not really helpful and Constructor was hardly magic in the first place.

Also, in the context of human mistakes, it is rather easy for this function to be modified, whereas the constructor could do nothing but to build a value of the type.

Lastly, partially exposing a type does not mean that we want to restrict pattern matching.

I’m kind of flying blind here, since there hasn’t been any concrete examples

I have given code that captures the spirit of the use case that I described, for the sake of simplicity. If you want to see “concrete” code, you can replace Some, Many and ALOT by product types, for instance.

The reason why downstream modules only need/want to handle some of these product types is that not every application is meant to market/control every product, just some of them. Adding support for a specific product is a choice by the business.

you’re not ready to publish a package.

This behaviour doesn’t impact package publishers only.

[Qqwy]

So, if I understand this correctly, it means that adding any new type constructor in your library will now result in the code of all your library’s consumers that happen to use the type constructor directly (rather than prefixing it with the module name) randomly breaking because of sudden naming conflicts?

[mattpiz]

Also I just read in the testing thread in slack that someone is having an issue with test dependencies because one package update had a minor update from 0.18 to 0.19 and elm-test picked the 0.18 version which caused the issue. Just to say, 0.19 might be a good excuse to say, “we need a breaking change” so we might as well improve the package design and make those custom types private. (I just hope you won’t have the package doc limitation again).

[Qqwy]

@bChiquet

If you allow me to reframe this in the context of our discussion, constructor = Constructor is not really helpful and Constructor was hardly magic in the first place.

This is exactly what I am thinking. For all intents and purposes, constructors are functions.
(The fact that Elm allows constructors in pattern matches while it does not allow other (bidirectional) functions does not diminish this fact, because it could; just like that only using integers does not suddenly change π(pi) from 3.14159265... to 3.)

[mgold]

Not unless someone else is already defining a value with that name. It’s the same as adding any other value to the package, if someone does a exposing (..).

[Qqwy]

Yes, but what this change is doing, is forcing the exposing (..) on the user.

I would say that it is relatively common for certain names to appear, and therefore naming conflicts to happen because of this change. Url, Request, Table, Color, Config, Encoder, Decoder, Key or even Type are very common names that immediately come to mind.