Experiment with Program Combinators

[rupert]

Reminds me of my take on a Middleware pattern: https://github.com/Janiczek/middleware

Thanks for the link, I was looking for other examples of Elm middleware.

There is also this one: https://github.com/RJWoodhead/elm-memo

[rupert]

Reminds me of my take on a Middleware pattern: https://github.com/Janiczek/middleware … It seemed to be a success but I didn’t get much echo from the community and didn’t try more middleware examples to find whether this is actually usable.

Going to have a play around with it, its not so easy to understand without trying out an example program or two.

[rupert]

Reminds me of my take on a Middleware pattern: https://github.com/Janiczek/middleware

Just to report back how I got on with this. I was fairly easily able to wrap my stateful auth module up as a Middleware. As with my combinators attempt, doing so saves a very small amount of boiler plate when integrating it. I am inclined to think, so what? It really isn’t that bad to just present it as a model/msg/update + API functions, and let the consumer wire it up with a small amount of boilerplate.

Your Middleware system seems more scalable than my attempt, in that you form them into a stack. However, capability is currently limited at compose3.

Is there some general compose : Middleware -> Middleware -> Middleware that could be written? Allowing as many as you like to be formed into a stack?

[evancz]

I’ve been seeing lots of talk about this kind of thing. I cannot participate, but I want to share a question I think about a lot:

How much complexity would you add to save five lines?

This question has become more and more important to me as I get more experience. As the Elm compiler and build tools mature, as I read more code by others, as I think about features, etc.

Another phrasing of this question is:

Why should all code be right at the border of human comprehension?

Sometimes code is just easy to understand. Humans could understand it if it was made more complex, but is that enough justification to do so? By now, I have improved code by going the opposite direction so many times that I am quite a skeptic.

Again, I cannot chat, but I wanted to share this perspective!

[rupert]

Why should all code be right at the border of human comprehension?

So people can’t reverse engineer my work so easily…?

Sometimes code is just easy to understand. Humans could understand it if it was made more complex, but is that enough justification to do so? By now, I have improved code by going the opposite direction so many times that I am quite a skeptic.

Again, I cannot chat, but I wanted to share this perspective!

Thanks for sharing that.

In OO programming we tend to encapsulate things in order to make them appear simpler from the outside. For example, a date picker widget just needs methods to show(), hide(), getDate(callback(Date date)), but internally it has all the details of how it is rendered, and what internal events it must process and so on; simple on the outside, complex on the inside.

Another example might by Arrays in Elm; outwardly they work like indexed arrays, inside they are actually pretty complex (https://github.com/elm-lang/core/blob/5.1.1/src/Native/Array.js#L1). The great thing is, in order to use Arrays, I don’t need to understand how they work, all I need is a simple mental model of an indexed look up.

Re-use of a piece of code is also a factor in this. If something is just being used in one project, it may not require the same level of thought going into its API, just wire it up and get your project done. Once something is being published as a package, you want to find a small, neat API that hides complexity behind it and provides a predictable mental model to work with.

Its different in functional programming to OO. I still think the same principals are at work, its just that they are expressed differently in FP code. As you are right to point out, in this case I think too much of an OO idea is being squeezed into an FP paradigm, causing an escalation in complexity that negates the original intention of trying to do it.

I hope I am not creating unwanted noise or distraction with my explorations - at the moment I am trying to explore lots of different ideas with Elm, and am also prepared to try out ideas and to be honest about the ones that are dead ends. Like this one.

So to report back on how this went - it probably isn’t such a useful pattern because it doesn’t scale.

Yeah, KISS - Keep It Simple Stupid.

[evancz]

I recommended my Accidentally Concurrent talk about how your two examples (an OO date picker vs a pure and immutable array library) are much less similar than they appear. Making boundaries and making boundaries around mutable state lead to incredibly different results!

And yeah, I don’t want to discourage exploration. I think it is important to explore! But I also do not want folks to get too optimistic about what is out there to find. So far I am pretty sure that the lambda calculus is nice, and that union types are a good idea. Past that I’m not as sure I find that this helps me focus my explorations, and the excitement of “I found something so simple it is surprising” is the best feeling! It’s rare though!

[MarkHamburg]

If you read Structure and Interpretation of Computer Programs (or go watch the lectures online), it comes back over and over again to building abstractions often with the idea that “here’s the API I want to program against” with the implementation of that API left to some hypothetical other programmer — often oneself. Too much code starts the other way in that it starts with doing something and then tries to modularize it or otherwise wrap an API around it based on what it does rather than what clients want it to do. API design patterns whether OO or FP including Elm’s (now somewhat disparaged as a structuring tool) MVU triplets provide pressure against that because they basically say “here is a way to modularize your code driven by an interface structure that is good for clients.”

Statically typed languages can be used to rule out various states that simply should not happen, but they can also slow down coding compared to dynamically typed languages and some of the constructions needed to achieve type safety can significantly raise the syntactic complexity of the code. If we want people to get started fast, then maybe JavaScript is a good choice. (Okay. Read Crockford and it will be clear why JavaScript is just loaded with land mines but there are dynamically-typed languages that contain JavaScript’s good parts without its bad parts. They just aren’t the language used by web browsers.)

The problem with program combinators is that they represent a level of cognitive indirection. They cease to be about building things by putting pieces — be they OO or functional — together and more about the notion of putting stuff together. In data, while it’s true that a record is a product type of its constituent elements, we’d rather just think about a record as having some fields. While programmers tend to love levels of indirection — “there is no problem that can’t be solved with another level of indirection” — in practice, they tend to grow uncomfortable with indirection in much the same way that non-programmers do.

So, the questions to ask should focus on whether there are patterns that are comprehensible — mentally and syntactically — to programmers that can guide construction of separable pieces of a program and which can then enable equally comprehensible composition of those separate pieces.

Mark

[rtfeldman]

Elm’s (now somewhat disparaged as a structuring tool) MVU triplets provide pressure against that because they basically say “here is a way to modularize your code driven by an interface structure that is good for clients.

I have to disagree! There has never been a time when MVU triplets could be described as either “good for clients” (they have always been useful in some situations and detrimental in others) or a good idea as a structuring tool (where they have always been detrimental).

[MarkHamburg]

The point to an API pattern is that by following it, clients know how to use the module without deep study of the particular subtleties of the module. For collections, map and filter represent API design patterns. For monadic constructs particularly those that can be thought of as sequencing andThen is an API design pattern. The message is that if you are building a collection or other wrapper type, look at these functions and try to follow the pattern in designing your own API.

In the old Elm programming guide for 0.17 and before that 0.16, MVU triplets provided a way to structure programs that allowed pieces to be combined without concerns as to the specifics of the pieces. If you wanted to put A and B side-by-side, if A and B both followed the MVU pattern, then the logic to put them side-by-side was straightforward. Did you need a list of C’s? Again, there were straightforward patterns to do so assuming that C followed MVU. The ability to wrap an MVU program to add navigation support again established that the pattern had structuring value. The downside was generally that while the code was straightforward to use and combine MVU modules, it wasn’t small and it got bigger with the addition of subscriptions. Program combinators as a concept `seem targeted at reducing that boiler plate coding but at this point also seem to come at the expense of abstracting away too much.

Now, where I would agree that MVU falls down is that including the view function in it overly ties code built using the API paradigm to the actual presentation. Strip it away to just MU (model&message + update, so maybe MMU) and you have a pattern for how to build a reusable element that needs to manage state and interact with the outside world. State transitions are driven by messages. Requests for actions in the outside world are delivered through commands. Again, we really need to throw in subscriptions as well some of the time, but something like Rupert’s auth module by following this pattern becomes an element that can be cleanly embedded in a variety of contexts without having to understand a lot of specifics about that particular embedding. But this also has little or nothing to do with the HTML being generated so, yes, thinking always in terms of view functions generating HTML is a bad idea. On the other hand, if we think of a view or public state function that provides a presentation to the outside world of the relevant state within the model, then we are back to MVU but in a way that doesn’t tie it tightly to the view system.

MVU worked as a pattern for building user interface centric code in Elm 0.16 and 0.17 — and as noted in other threads, Richard, you’ve used it yourself for SPA pages — but in part because the examples were kept small for clarity and in part because of baggage from other languages, people built stateful elements with their own MVU triples where no state management was really needed. The cure for this was pointing out that often a view function — an even simpler API pattern — was sufficient and even easier to reuse. The greatest tension I’ve seen from the standpoint of building more complex programs from smaller pieces is when the natural data model does not match the desired presentation. Here MVU was a distinct negative because of its focus on HTML views. But applying MU to the data model and then considering how to transform the natural data model into the form that fits the needs of the view — see, for example, Charlie Koster’s Medium piece on selectors — seems to point the way to resolving this tension.

Mark

[rtfeldman]

The cure for this was pointing out that often a view function — an even simpler API pattern — was sufficient and even easier to reuse.

To me, the following has always been the best advice: “build the simplest API that gets the job done.”

Historically, a common pitfall has been to choose a “one size fits all” API (nested MVU), and then only after using it pervasively to discover that this one API design did not actually “fit all” after all. Whoops.

Avoiding this pitfall requries not thinking in terms of “I must find the silver bullet API that will solve all my problems if I use it everywhere” but rather in terms of “I’m gonna build the simplest API that gets the job done for the use case at hand.” These one-size-fits-all silver bullets have never existed, and “maybe MU is the answer” misses the point precisely as much as “maybe MVU is the answer” does.

The answer is building the simplest API that gets the job done! And the simplest API that gets the job done will inherently vary on a case-by-case basis.

[pdamoc]

There is no real alternative to the MVU triplet where accidental state is involved.

In some cases we can pretend that we don’t do the MVU triplet by doing it in a funny way (elm-sortable-table) but that trick works only if one does not need Cmds and it only shaves a small part of the boilerplate.

The consequence of this is that Elm does not have any kind of officially blessed UI toolkit and all attempts to create such a toolkit end up in a lot of troubles.

If a beginner asks for some sort of UI toolkit what should we tell him? elm-ui ? That’s not in the elm-packages. elm-mdl? The author bailed out and it is built on an obsolete technology. elm-bootstrap? Well, guess what… it’s using the MVU pattern and building a complex enough form will bring up the issue of boilerplate.

A “simple” use case of a small SPA with few pages, a nav bar that collapses into a hamburger menu and a complex enough form (few dropdowns and some datepickers) is enough to outline this problem.

This is a very complex issue. Also, the cure proposed by the OP looks to me way worse than the disease. Sorry, but I cannot begin to contemplate how could I explain this to a beginner.

[rtfeldman]

There is no real alternative to the MVU triplet where accidental state is involved. In some cases we can pretend that we don’t do the MVU triplet by doing it in a funny way (elm-sortable-table)

This is like saying immutability is a funny way to pretend we don’t do mutability. There’s nothing “funny” about choosing not to overcomplicate an API - that’s just good design!

elm-bootstrap? Well, guess what… it’s using the MVU pattern and building a complex enough form will bring up the issue of boilerplate.

Do you think elm-bootstrap overuses MVU? If so, in which modules?

Separately, do you think “forms involve boilerplate” (a true statement about forms, independent of language, library, or indeed computers!) is a good reason to discourage a beginner from using a library?

[pdamoc]

This is like saying immutability is a funny way to pretend we don’t do mutability. There’s nothing “funny” about choosing not to overcomplicate an API - that’s just good design!

MVU is still immutability and elm-sortable-table is still MVU. It is not classic MVU but it is still MVU. What happened is that this pattern got promoted as an alternative to MVU while still being MVU in disguise. This pattern also cannot handle the more complicated “widgets” that require Cmds or subscriptions.

Do you think elm-bootstrap overuses MVU? If so, in which modules?

All the widgets that require accidental state are using MVU. I’m not saying that it is overusing… just that it is unavoidable.

Separately, do you think “forms involve boilerplate” (a true statement about forms, independent of language, library, or indeed computers!) is a good reason to discourage a beginner from using a library?

I think that boilerplate is a sign of problems. It overcomplicates something that by its very nature is already complicated enough.

[rupert]

Also, the cure proposed by the OP looks to me way worse than the disease.

Lol. This one is a dead end for sure.

What I am learning is that there is no cure for the disease. The disease is trying to do OO in Elm, and the only way is to drink the kool aid and begin to see things from another perspective. If you have heart disease because you ate a bad diet your whole life then there is nothing inevitable about that outcome; your doctor will tell you to change your behavior before it is too late.

There is no real alternative to the MVU triplet where accidental state is involved.

Just to check, when you say accidental state here you don’t mean program states than can happen by accident, as per making impossible states impossible do you? You are meaning state that you want to make internal to some component because the consumer of that component doesn’t need to know about it?

elm-mdl? The author bailed out and it is built on an obsolete technology.

I hadn’t kept up with this story, was there some communication on the elm-discuss list where he quit and gave reasons why? Also, when you say its obsolete, which aspect? The MDL-light stuff published by Google is also abandonware? or the whole MDL concept is now dead?

I ask because I am still using elm-mdl and think its pretty convenient, but I need to think again if it is deprecated.

[pdamoc]

Just to check, when you say accidental state here you don’t mean program states than can happen by accident, as per making impossible states impossible do you? You are meaning state that you want to make internal to some component because the consumer of that component doesn’t need to know about it?

The easiest way to think about this is by looking at the input tag. It has behavior, it has internal state. you can move the cursor, you can do a selection inside. None of this is forced on the user, you don’t have to pass around some internal state for that widget because it is managed by the browser inside the low level widget that it is based on. The state of this widget is accidental to the user of the widget. If you want to read more about this use of accidental state you can read Out of the tar pit.

I hadn’t kept up with this story, was there some communication on the elm-discuss list where he quit and gave reasons why? Also, when you say its obsolete, which aspect? The MDL-light stuff published by Google is also abandonware? or the whole MDL concept is now dead?

It looks like I spoke too soon. I was under the impression that Søren was no longer actively developing the library but it seams that he still works on it. elm-mdl is based on material-desing-lite which was abandoned by google in favor of a new approach: material-components-web. Material design as a design philosophy is here to say for the foreseeable future.

[rupert]

If you want to read more about this use of accidental state you can read Out of the tar pit.

I’ll give that a read later, but I honestly think it doesn’t say anything beyond what is already neatly summed up by Parnas’ principles.

Taking elm-mdl as an example again, it has accidental state which it keeps in it its Material.Model type. I just have to remember to hook up its update function, which really isn’t so bad.

As far as I can tell, there are only 2 ways that the boilerplate hooking up of the update function could be avoided. The first would be allow mutable state. The second would be to allow the Elm runtime to hold >1 model at the top level, each with an independent Program and then to allow programs to communicate via messages.

Option 2 was what I was aiming for with this experiment. Even if that were achievable, I have now come to the conclusion that it is generally not going to be a productive way of programming. The reason being, that if components communicate with events rather than having a single model, they tend to end up modelling some of each others state, and duplicated state is duplicated effort as well as somewhere for bugs to hide.

Is there some other option that it better?

[rupert]

Hope I’m not going on too much with the tl;dr…

It comes down to the essential difference between these 2 pieces of code:

class SomeClass {
    String state;

    void update(String val) { this.state = state }
}

and

type alias SomeClass =
    { state:String
    }

update : String -> SomeClass -> SomeClass
update val this = { this | state = val }

Snippet one produces a new object and overwrites the old one with it. Snippet two produces a new object, but leaves it up to the caller to decide where to put the new one.

In that sense immutable FP is more powerful than OO because we don’t always have to take those 2 actions together.

If we are trying to take something away from this that could improve Elm, it might be that a little bit of syntactic sugar to help with updating nested records could be a welcome addition.

[pdamoc]

Is there some other option that it better?

There is no simple option. If it would have been it would have been discovered by now.

From my own investigation into this subject, there could be a way out of this by some kind of clever datastructure that would augment the current virtual-dom nodes with the possibility of declaring your own web-components-like structures. This is however no simple matter and a lot of thought needs to go into it.

[MarkHamburg]

A partial solution for accidental widget state would be a concerted effort to make sure web components work well with the virtual DOM including making sure they aren’t accidentally created, destroyed, or reused because the DOM diff either didin’t recognize continuity of identity or misrecognized it. It’s not hard to implement Elm code that exhibits the continuity issues with input fields. (Misrecognition is always addressable by using Html.Keyed but since there is only a one element look ahead in the DOM diff, Html.Keyed isn’t always sufficient to preserve identity and hence accidental state.)

A further solution would look at how to write components using Elm presumably as something like independent MVU programs.

I know people have demonstrated proofs of concept for these things but to really call it a solution takes pounding through the issues to eliminate the subtle dangers.

Of course, the first step in solving a problem is admitting that you have one. In this case, it’s admitting that accidental state is a thing and we are already using it every time we create an input field.

Mark

[MarkHamburg]

To me, the following has always been the best advice: “build the simplest API that gets the job done.”

You know what’s simple? Go to statements are simple, The structured programming strictures against using go to statements made programs harder to write — i.e., more complicated — because you couldn’t do things like just exit out wherever you wanted to.

You know what’s simple? Monolithic programs and global variables rather modularized programs and heap allocated data. Modularized programs require going through APIs to get to data and you have to pass around more data references when you don’t just use global variables. Monolithic programs with global variables generally have significantly simpler code.

You know what’s simple? Mutability is simple. How do I increase the count of the number of cats? Increment the cats counter is a lot simpler than return a new model with a different value for the cats counter.

I am actually all for simplicity. I prefer C over C++. I prefer Scheme over Common LISP. I prefer Elm’s semantics over ML’s. But I also know that “simplicity” is often an argument for all sorts of practices that have proven problematic over time when looked at in the context of building large, reliable programs.

Mark