Experiment with Program Combinators

Show and Tell
1

So to report back on how this went - it probably isn’t such a useful pattern because it doesn’t scale.

I started out by forming the Day convolution using joneshf/elm-comonad from its Html.Alternative module. Then I rewrote that as a combinator of more normal Elm Programs with init/model/update/subscriptions and a Msg type (joneshf/elm-comonad does away with the Msg type).

I introduced also the concept of a message channel between the 2 combined programs, which is very similar to an out message. The difference is that the 2 combined programs are not combined in a parent-child nested TEA pattern, but are paired together side by side, to form a new combined program.

Here is the code:

So I could use this to write my authentication module which has its own private internal state, as a HeadlessProgramWithChannel. Then I could write a user interface program as an HtmlProgramWithChannel. The messages going from the UI to the auth module would be a Msg type describing the actions it is requested to take (login/logout/unauthed). The messages coming back from the auth module would be a record type describing just a portion of the total authentication state - a status saying whether we are currently logged in or not, and if logged in what the user id and permission scopes are. The JWT token in particular would not be visible at all on the UI side, and there is other internal state too, such as a record of when the token expires, and a refresh token to obtain a new one, and so on.

The problem is that the combined program type is a product of the types of the programs it combines:

combineHeadlessAndHtmlWithChannel :
    HeadlessProgramWithChannel modela msga send recv
    -> HtmlProgramWithChannel modelb msgb recv send
    -> HtmlProgram ( modela, modelb ) (Msg msga msgb)

The only way I can think of to loosen the typing is to convert all messages and models to Json.Encode.Values. Then it would be possible to combine many programs in this way, as a kind of plug and play message bus, but at the cost of writing a lot of encoders/decoders.

It is interesting to compare with an OO language, where sub-typing allows types to be loosened. So in an OO language I could have a list of plugins (List Plugin) where each Plugin defines an interface and has its own implementing sub-type. At the level where the plugins are combined into a system, the typing is loosened sufficiently that they are all just plugins with a defined mechanism to plug them together. In Elm typing is much stricter, which rules this out, except by the mechanism of converting to Value. Its like trying to put a Plugin a and Plugin b into the same list in Elm; it won’t let you do it unless a and b are the same type. Or by having common global message and model types across the whole application which may defeat the purpose, and not all modules need to talk to each other.

I only have one auth module with state, all my other modules that talk to back-end APIs are stateless. So I could use the above combinator to integrate the auth module, and save myself the tiny hassle of adding a clause to the parent Msg type using Cmd.map to link in its update cycle to the application.

Is it worth it? Probably not, not without gaining something in terms of easier scalability.

2 Likes
2

Hi :slight_smile: I was looking over your post, and wondering what problem you are having, that the idea of combining top-level programs is going to solve?

In my experience, combining programs is straight-forward if you “separate the apples and oranges” of the program; model, update, subscriptions and view. I’m guessing you’re trying to achieve something else than combining top-level programs?

3

I re-use my stateful authentication module and other stateless API modules across >1 UI implementations. I’m looking for ways in which I can bundle and publish such things as packages, and make the integration with the package as simple as possible. If you unbundle it to model/update/subscription/view you must integrate each of those pieces independently; so we say that nesting TEA requires a certain amount of boilerplate. I was just experimenting to see if I can figure out something simpler, where the boilerplate aspect is taken care of by some combinator code.

1 Like
4

Reminds me of my take on a Middleware pattern: https://github.com/Janiczek/middleware … It seemed to be a success but I didn’t get much echo from the community and didn’t try more middleware examples to find whether this is actually usable.

5

Thanks for the link, I was looking for other examples of Elm middleware.

There is also this one: GitHub - RJWoodhead/elm-memo: An exploration of the memo metaphor for Elm module communications

6

Going to have a play around with it, its not so easy to understand without trying out an example program or two.

7

Just to report back how I got on with this. I was fairly easily able to wrap my stateful auth module up as a Middleware. As with my combinators attempt, doing so saves a very small amount of boiler plate when integrating it. I am inclined to think, so what? It really isn’t that bad to just present it as a model/msg/update + API functions, and let the consumer wire it up with a small amount of boilerplate.

Your Middleware system seems more scalable than my attempt, in that you form them into a stack. However, capability is currently limited at compose3.

Is there some general compose : Middleware -> Middleware -> Middleware that could be written? Allowing as many as you like to be formed into a stack?

8

I’ve been seeing lots of talk about this kind of thing. I cannot participate, but I want to share a question I think about a lot:

How much complexity would you add to save five lines?

This question has become more and more important to me as I get more experience. As the Elm compiler and build tools mature, as I read more code by others, as I think about features, etc.

Another phrasing of this question is:

Why should all code be right at the border of human comprehension?

Sometimes code is just easy to understand. Humans could understand it if it was made more complex, but is that enough justification to do so? By now, I have improved code by going the opposite direction so many times that I am quite a skeptic.

Again, I cannot chat, but I wanted to share this perspective!

7 Likes
9

So people can’t reverse engineer my work so easily…? :grin:

Thanks for sharing that.

In OO programming we tend to encapsulate things in order to make them appear simpler from the outside. For example, a date picker widget just needs methods to show(), hide(), getDate(callback(Date date)), but internally it has all the details of how it is rendered, and what internal events it must process and so on; simple on the outside, complex on the inside.

Another example might by Arrays in Elm; outwardly they work like indexed arrays, inside they are actually pretty complex (core/src/Native/Array.js at 5.1.1 · elm-lang/core · GitHub). The great thing is, in order to use Arrays, I don’t need to understand how they work, all I need is a simple mental model of an indexed look up.

Re-use of a piece of code is also a factor in this. If something is just being used in one project, it may not require the same level of thought going into its API, just wire it up and get your project done. Once something is being published as a package, you want to find a small, neat API that hides complexity behind it and provides a predictable mental model to work with.

Its different in functional programming to OO. I still think the same principals are at work, its just that they are expressed differently in FP code. As you are right to point out, in this case I think too much of an OO idea is being squeezed into an FP paradigm, causing an escalation in complexity that negates the original intention of trying to do it.

I hope I am not creating unwanted noise or distraction with my explorations - at the moment I am trying to explore lots of different ideas with Elm, and am also prepared to try out ideas and to be honest about the ones that are dead ends. Like this one.

Yeah, KISS - Keep It Simple Stupid.

10

I recommended my Accidentally Concurrent talk about how your two examples (an OO date picker vs a pure and immutable array library) are much less similar than they appear. Making boundaries and making boundaries around mutable state lead to incredibly different results!

And yeah, I don’t want to discourage exploration. I think it is important to explore! But I also do not want folks to get too optimistic about what is out there to find. So far I am pretty sure that the lambda calculus is nice, and that union types are a good idea. Past that I’m not as sure :stuck_out_tongue: I find that this helps me focus my explorations, and the excitement of “I found something so simple it is surprising” is the best feeling! It’s rare though!

6 Likes
11

If you read Structure and Interpretation of Computer Programs (or go watch the lectures online), it comes back over and over again to building abstractions often with the idea that “here’s the API I want to program against” with the implementation of that API left to some hypothetical other programmer — often oneself. Too much code starts the other way in that it starts with doing something and then tries to modularize it or otherwise wrap an API around it based on what it does rather than what clients want it to do. API design patterns whether OO or FP including Elm’s (now somewhat disparaged as a structuring tool) MVU triplets provide pressure against that because they basically say “here is a way to modularize your code driven by an interface structure that is good for clients.”

Statically typed languages can be used to rule out various states that simply should not happen, but they can also slow down coding compared to dynamically typed languages and some of the constructions needed to achieve type safety can significantly raise the syntactic complexity of the code. If we want people to get started fast, then maybe JavaScript is a good choice. (Okay. Read Crockford and it will be clear why JavaScript is just loaded with land mines but there are dynamically-typed languages that contain JavaScript’s good parts without its bad parts. They just aren’t the language used by web browsers.)

The problem with program combinators is that they represent a level of cognitive indirection. They cease to be about building things by putting pieces — be they OO or functional — together and more about the notion of putting stuff together. In data, while it’s true that a record is a product type of its constituent elements, we’d rather just think about a record as having some fields. While programmers tend to love levels of indirection — “there is no problem that can’t be solved with another level of indirection” — in practice, they tend to grow uncomfortable with indirection in much the same way that non-programmers do.

So, the questions to ask should focus on whether there are patterns that are comprehensible — mentally and syntactically — to programmers that can guide construction of separable pieces of a program and which can then enable equally comprehensible composition of those separate pieces.

Mark

1 Like
12

I have to disagree! There has never been a time when MVU triplets could be described as either “good for clients” (they have always been useful in some situations and detrimental in others) or a good idea as a structuring tool (where they have always been detrimental).

13

The point to an API pattern is that by following it, clients know how to use the module without deep study of the particular subtleties of the module. For collections, map and filter represent API design patterns. For monadic constructs particularly those that can be thought of as sequencing andThen is an API design pattern. The message is that if you are building a collection or other wrapper type, look at these functions and try to follow the pattern in designing your own API.

In the old Elm programming guide for 0.17 and before that 0.16, MVU triplets provided a way to structure programs that allowed pieces to be combined without concerns as to the specifics of the pieces. If you wanted to put A and B side-by-side, if A and B both followed the MVU pattern, then the logic to put them side-by-side was straightforward. Did you need a list of C’s? Again, there were straightforward patterns to do so assuming that C followed MVU. The ability to wrap an MVU program to add navigation support again established that the pattern had structuring value. The downside was generally that while the code was straightforward to use and combine MVU modules, it wasn’t small and it got bigger with the addition of subscriptions. Program combinators as a concept `seem targeted at reducing that boiler plate coding but at this point also seem to come at the expense of abstracting away too much.

Now, where I would agree that MVU falls down is that including the view function in it overly ties code built using the API paradigm to the actual presentation. Strip it away to just MU (model&message + update, so maybe MMU) and you have a pattern for how to build a reusable element that needs to manage state and interact with the outside world. State transitions are driven by messages. Requests for actions in the outside world are delivered through commands. Again, we really need to throw in subscriptions as well some of the time, but something like Rupert’s auth module by following this pattern becomes an element that can be cleanly embedded in a variety of contexts without having to understand a lot of specifics about that particular embedding. But this also has little or nothing to do with the HTML being generated so, yes, thinking always in terms of view functions generating HTML is a bad idea. On the other hand, if we think of a view or public state function that provides a presentation to the outside world of the relevant state within the model, then we are back to MVU but in a way that doesn’t tie it tightly to the view system.

MVU worked as a pattern for building user interface centric code in Elm 0.16 and 0.17 — and as noted in other threads, Richard, you’ve used it yourself for SPA pages — but in part because the examples were kept small for clarity and in part because of baggage from other languages, people built stateful elements with their own MVU triples where no state management was really needed. The cure for this was pointing out that often a view function — an even simpler API pattern — was sufficient and even easier to reuse. The greatest tension I’ve seen from the standpoint of building more complex programs from smaller pieces is when the natural data model does not match the desired presentation. Here MVU was a distinct negative because of its focus on HTML views. But applying MU to the data model and then considering how to transform the natural data model into the form that fits the needs of the view — see, for example, Charlie Koster’s Medium piece on selectors — seems to point the way to resolving this tension.

Mark

2 Likes
14

The cure for this was pointing out that often a view function — an even simpler API pattern — was sufficient and even easier to reuse.

To me, the following has always been the best advice: “build the simplest API that gets the job done.”

Historically, a common pitfall has been to choose a “one size fits all” API (nested MVU), and then only after using it pervasively to discover that this one API design did not actually “fit all” after all. Whoops.

Avoiding this pitfall requries not thinking in terms of “I must find the silver bullet API that will solve all my problems if I use it everywhere” but rather in terms of “I’m gonna build the simplest API that gets the job done for the use case at hand.” These one-size-fits-all silver bullets have never existed, and “maybe MU is the answer” misses the point precisely as much as “maybe MVU is the answer” does.

The answer is building the simplest API that gets the job done! And the simplest API that gets the job done will inherently vary on a case-by-case basis.

4 Likes
15

There is no real alternative to the MVU triplet where accidental state is involved.

In some cases we can pretend that we don’t do the MVU triplet by doing it in a funny way (elm-sortable-table) but that trick works only if one does not need Cmds and it only shaves a small part of the boilerplate.

The consequence of this is that Elm does not have any kind of officially blessed UI toolkit and all attempts to create such a toolkit end up in a lot of troubles.

If a beginner asks for some sort of UI toolkit what should we tell him? elm-ui ? That’s not in the elm-packages. elm-mdl? The author bailed out and it is built on an obsolete technology. elm-bootstrap? Well, guess what… it’s using the MVU pattern and building a complex enough form will bring up the issue of boilerplate. :slight_smile:

A “simple” use case of a small SPA with few pages, a nav bar that collapses into a hamburger menu and a complex enough form (few dropdowns and some datepickers) is enough to outline this problem.

This is a very complex issue. Also, the cure proposed by the OP looks to me way worse than the disease. Sorry, but I cannot begin to contemplate how could I explain this to a beginner.

2 Likes
16

This is like saying immutability is a funny way to pretend we don’t do mutability. There’s nothing “funny” about choosing not to overcomplicate an API - that’s just good design!

Do you think elm-bootstrap overuses MVU? If so, in which modules?

Separately, do you think “forms involve boilerplate” (a true statement about forms, independent of language, library, or indeed computers!) is a good reason to discourage a beginner from using a library?

17

MVU is still immutability and elm-sortable-table is still MVU. It is not classic MVU but it is still MVU. What happened is that this pattern got promoted as an alternative to MVU while still being MVU in disguise. This pattern also cannot handle the more complicated “widgets” that require Cmds or subscriptions.

All the widgets that require accidental state are using MVU. I’m not saying that it is overusing… just that it is unavoidable.

I think that boilerplate is a sign of problems. It overcomplicates something that by its very nature is already complicated enough.

18

Lol. This one is a dead end for sure.

What I am learning is that there is no cure for the disease. The disease is trying to do OO in Elm, and the only way is to drink the kool aid and begin to see things from another perspective. If you have heart disease because you ate a bad diet your whole life then there is nothing inevitable about that outcome; your doctor will tell you to change your behavior before it is too late.

Just to check, when you say accidental state here you don’t mean program states than can happen by accident, as per making impossible states impossible do you? You are meaning state that you want to make internal to some component because the consumer of that component doesn’t need to know about it?

I hadn’t kept up with this story, was there some communication on the elm-discuss list where he quit and gave reasons why? Also, when you say its obsolete, which aspect? The MDL-light stuff published by Google is also abandonware? or the whole MDL concept is now dead?

I ask because I am still using elm-mdl and think its pretty convenient, but I need to think again if it is deprecated.

1 Like
19

The easiest way to think about this is by looking at the input tag. It has behavior, it has internal state. you can move the cursor, you can do a selection inside. None of this is forced on the user, you don’t have to pass around some internal state for that widget because it is managed by the browser inside the low level widget that it is based on. The state of this widget is accidental to the user of the widget. If you want to read more about this use of accidental state you can read Out of the tar pit.

It looks like I spoke too soon. I was under the impression that Søren was no longer actively developing the library but it seams that he still works on it. elm-mdl is based on material-desing-lite which was abandoned by google in favor of a new approach: material-components-web. Material design as a design philosophy is here to say for the foreseeable future.

1 Like
20

I’ll give that a read later, but I honestly think it doesn’t say anything beyond what is already neatly summed up by Parnas’ principles.

Taking elm-mdl as an example again, it has accidental state which it keeps in it its Material.Model type. I just have to remember to hook up its update function, which really isn’t so bad.

As far as I can tell, there are only 2 ways that the boilerplate hooking up of the update function could be avoided. The first would be allow mutable state. The second would be to allow the Elm runtime to hold >1 model at the top level, each with an independent Program and then to allow programs to communicate via messages.

Option 2 was what I was aiming for with this experiment. Even if that were achievable, I have now come to the conclusion that it is generally not going to be a productive way of programming. The reason being, that if components communicate with events rather than having a single model, they tend to end up modelling some of each others state, and duplicated state is duplicated effort as well as somewhere for bugs to hide.

Is there some other option that it better?