Dynamic record updates

[skosch]

Hi all,

I’m relatively new to Elm and I really love it so far – except, and this is becoming a dealbreaker for me, the inability to make dynamic updates to records, i.e. not having to write dozens of separate, 100%-identical-looking setter functions for Every. Single. Field.

I’m aware that a solution would require some code generation – it doesn’t have to be a full-scale user-facing macro system. I’m aware of extensible records and how to use them. I’m aware of the Focus library (and that it does nothing to address the actual issue). I’m aware of Dicts (but then I might as well not use a type-checked language at all). I’ve read all the mailing list threads and Github issues. I’m aware that this has been requested over and over for years, and shot down every time because of concerns new developers might get confused and/or write messy code.

Is there something in the pipeline to make this happen – a discussion I could follow, a dev branch I can look at and contribute to, a workaround, anything? I’d hate to leave Elm, but it’s turning me into a human templating system and it’s driving me mad.

Thanks! With love,
Sebastian

[folkertdev]

Can you elaborate with a piece of code where the lack of dynamic setters is problematic?
How would derived setters solve the problem?

In terms of length, adding setters would have very little gain

-- current 
{ record | fieldName = newValue } 

-- alternative
record.setFieldname newValue -- possibly leads to name clashes and adds magic
record@fieldname newValue -- not very pretty

So I guess the goal is composition of setters? The elm-focus documentation already describes why nested
data updates in pure functional languages - and possibly in general - may not be a great idea.

Unmentioned there is that without serious compiler optimizations, setters generate much more garbage, and
are thus slower

-- creates one intermediate record, and then the new one
myRecord
    |> setX 10
    |> setY 20

-- creates just the new record
{ myRecord | x = 10, y = 20 } 

You acknowledge that this has been suggested before, and was treated with scepticism.
I think there are good reasons for that scepticism - both stylistic and practical.
Flipping the question: Why do we need this feature. How would our applications and the language in general
become better with this change.

[skosch]

Thanks @folkertdev for the quick response.

One example: I often find myself writing

updateEmail : String -> Model -> Model
updateEmail email model =
  { model | email = email}

updateAddress : String -> Model -> Model
updateAddress address model =
  { model | address = address}

...

They look exactly the same, and I have way too many of those. And then you need corresponding Msg types (or giant case-of trees). Extensible records can tighten things up but they don’t fix duplication. It really rubs me the wrong way, and no argument about “simplicity” or “readability” or “explicit-ness” is going to convince me otherwise. It’s 2018, I’m not a beginner, I shouldn’t have to copy-paste code.

Instead, something like (don’t quote me on the syntax)

updateField : Key<Model> -> Model -> Value<Model> -> Model
updateField field model value =
  { record | ^field = value }

could get auto-expanded based on Model's definition, and would allow you to make quick one-shot updates effectively directly from your view – you would only need a single Msg type that captures what field you’re trying to update, and with what value.

Your reaction sounds like there is little appetite for this kind of thing, though. Maybe I should be looking at GHCJS instead?

[folkertdev]

But why do you have these setters in the first place? what is better about them than a literal record update.

{ model | email = newEmail, address = newAddress } 

is actually shorter than

model |> updateEmail newEmail |> updateAddress newAddress

[skosch]

Right – the underlying issue is that I have UI with lots of repeating or dynamically-displayed components, and I would like their e.g. onInput messages to all look the same, like

  input [ onInput <| UpdateField ^email ] []
, input [ onInput <| UpdateField ^address ] []

In other words, I don’t want to maintain an UpdateEmail message and an UpdateAddress message, but only one UpdateField message that includes the key. Does that make sense?

(Edit: removed the extra model and added ^ to distinguish field names)

[folkertdev]

Yes!

Now we have a concrete problem we can attack.

Sadly, there are again no easy answers. the type of ^email would be String -> { a | email : String } -> { a | email : String }. The type of UpdateField in the most general case is quite hairy and certainly not expressible in current elm types. (It would have to use universal quantification if that is something you’re familiar with). Even if we restrict to

type Msg = UpdateField (String -> Model -> Model)

There is now a function in our message type. This means the model is no longer serializable. This means the debugger stops working and some of your update logic is now actually outside of the update function.

All of this is to say: this stuff is complicated. Some of the problems can be fixed (custom serialization would make the debugger work again) but the fixes open up more cans of worms (typeclasses). Lifting update logic out of the update function is a more fundamental problem. Language design is incredibly hard, and every decision comes with a trade-off.

Elm has made some choices, and these influence what we can do. Haskell has made different choices, purescript has made different choices. So by all means see how haskell has solved this problem, and in particular purescript, with its much more advanced record system, might be a better fit for you for this project.

[christian]

In other words, I don’t want to maintain an UpdateEmail message and an UpdateAddress message, but only one UpdateField message that includes the key. Does that make sense?

One way we’ve accomplished something like this is to wrap various related update actions in a single Msg by passing it a secondary type that acts as a ‘sub-Msg’.

type Msg
    = UpdateField UpdateFieldMsg
    | ...

type UpdateFieldMsg
    = UpdateEmail String
    | UpdateAddress String
    | UpdateAge Int

update : Msg -> Model -> (Model, Cmd Msg)
update msg model =
    case msg of
        UpdateField updateFieldMsg ->
            updateField updateFieldMsg model

updateField : UpdateFieldMsg -> Model -> (Model, Cmd Msg)
updateField updateFieldMsg model =
    case updateFieldMsg of
        UpdateEmail email ->
            ...


-- in your View

input [ onInput <| UpdateField << UpdateEmail ] []

This essentially allows you to separate your update function into a sub-functions that bundle related field updates. You are not writing less code, but you can bundle your code together. I realize this does not solve the original problem, which my team has encountered as well. I would like to see a solution that does not involve writing tons of boilerplate code or using a module that has “not recommended” written all over it.

[skosch]

Thank you both! The Elm community certainly is wonderful

@folkertdev You say this stuff is complicated – and I totally get that. It looks like I’m part of a small (if vocal) minority that will gladly deal with complex abstraction magic if it means reduced code duplication. If that’s not what Elm is aiming for, then I may need to look elsewhere.

[rtfeldman]

I don’t want to maintain an UpdateEmail message and an UpdateAddress message, but only one UpdateField message that includes the key. Does that make sense?

This used to really bother me too! Here’s how I solved it:

1. I reflected on all the time I’ve spent in my career trying to reduce irritating boilerplate code - which by now I’ve done in at least a dozen languages.
2. I thought about how complex the code was, and how easy it was to work with, both before and after the changes I made in the name of reducing boilerplate.
3. I wanted my time back.

When repetitive code is error-prone, that’s a problem I want to fix. But when it’s merely tedious, like the code above, I’ve found it’s easy for me to fall into the trap of spending 20 hours questing for a way to save 45 minutes of repetitive coding over the life of the project.

In retrospect, these quests have had ludicrously negative payoffs in aggregate - especially because my solutions tended to be woefully overcomplicated compared to the drearily simple boilerplate - but it was admittedly more fun than the more time-efficient solution of writing the unfancy boilerplate and moving on.

Anyway, once I reflected back on the outcomes of my many past quests to reduce boilerplate, it changed my perspective.

Now when I see some code that’s easy to maintain but tedious to write, and I start to think “there must be a way to get rid of these few lines of boilerplate, if I just spend a few more weekends of my life searching for The Answer, or maybe if I hit up the designer of the library/framework/language to make this one design change that will solve it with surely no unanticipated downsides” my next thought is “oh, I’ve seen this trap before - I’m about to sink a bunch of time into something I’ll regret having spent this much time on later.”

Then I think "this is maintainable, straightforward code, and I’m going to keep it that way." And I move on.

YMMV, but I thought I’d share how I got to a point where this did not bother me anymore! It’s possible you’ve been much more successful at this than I have, but I think the self-reflection exercise was well worth the time, and I’d encourage anyone to take an honest self-assessment of how these quests have turned out in the past.

[skosch]

… and yet, here you are, wasting all that precious reclaimed time patronizing internet strangers!

I kid. Of course you’re right. The crux is, if we’re talking life philosophy, then to me web dev may be a remarkably comfortable way to pay the bills, but it’s also just really mundane, boring labour. I’m self-employed mostly so I get to choose my own tools (good riddance, Java EE), and what keeps me going are the little moments of giggly excitement about magical, zen-like elegance. Elm has given me a good number of those, but I really don’t know if I can get over its verbosity.

I am going to do my best to adopt your mindset though – thanks for your well-written reply.

[Sidney]

YMMV, but I thought I’d share how I got to a point where this did not bother me anymore! It’s possible you’ve been much more successful at this than I have, but I think the self-reflection exercise was well worth the time, and I’d encourage anyone to take an honest self-assessment of how these quests have turned out in the past.

I totally see where you’re coming from. I had a similar realization, the tools/applications/enhancements I attempt to make seem to take more time than the original task would have! But that doesn’t stop me from trying

Lately, instead of sinking 20 hours into some improvement that isn’t even guaranteed to help, I’ll just spend an hour or two. If I like what I come up with, I’ll give it a shot. Maybe it isn’t perfect, but I try to improve it over time. Maybe I hate it – I likely didn’t fully understand the problem. In that case, I’ll just do the task when I need to, but continue to think in the back of my mind how it could be automated, or how the boilerplate could be reduced. Hopefully in the future I come up with something better.

---

Bringing it back to this thread’s topic, I think we have identified a problem (boilerplate in some circumstances). I’d like to consider solutions before conceding to live with it! I think the simplest solution is a record updater syntax, to complement the record accessor syntax. (I know this has been suggested before, and even in this thread!)

-- accessor
.x : { a | x : b } -> b

-- updater
^x : b -> { a | x : b } -> { a | x : b }

Now, is that perfect? Does it even help? No idea. I wish I could test it! If I knew more Haskell, I’d fork the compiler and add it in. At least to see how it feels to use. (Maybe the arguments should be flipped? Maybe it should use a different symbol?)

Another problem I saw mentioned in this thread is the idea of wiring up, for example, form fields. Every field is tracked in the model, and updates need to come back and get set in the model. Very boilerplatey.

JavaScript, for example, has a lot of ways to deal repetitive code like this. You can pass around field names (ie object keys) as strings, and access fields on the fly (obj[keyName]). That allows you to store your fields in some kind of set or list, and dynamically access the correct one. Of course, JavaScript can’t do this in a type-safe way… can we do this in Elm? Well, I think @hecrj’s form API idea/proposal is a very good start (I recently used it in a project and found it prevents a lot of this type of boilerplate). But I feel like we can do something more at a language/syntax level.

I have a vague Idea in my mind that macros of some kind might help, but I’m not sure what that would look like… personally, I’m still mulling over possible solutions, nothing solid at this point.

I’m eager to see what else we can come up with

[skosch]

I like the idea of the ^x updater function. It feels super convenient. @folkertdev is right though that putting functions into messages may break other tools that rely on serializability.

What you’d need is some sort of universal quantification/Typeable thing that @folkertdev hinted at. Consider a generic message type

type Msg = UpdateField forallKeys(Model, ^key) typeOf(Model, ^key)

This message type understands that its first parameter must be a field from Model, and its second parameter must be of the type of the corresponding value (for ^email, that’d be String). I’m not an expert, but I would conjecture that wouldn’t be too difficult to typecheck.

So I can then construct

myMsg : Msg
myMsg = UpdateField email "test@abc.com"

… and pattern match in my update module:

case message of
    UpdateField ^key value ->
        ( { model | ^key = value }, Cmd.none )

I’m using the ^ to show key variables – it’s not strictly necessary but I think it’s a helpful visual.

[pdamoc]

I’m relatively new to Elm and I really love it so far – except, and this is becoming a dealbreaker for me, the inability to make dynamic updates to records, i.e. not having to write dozens of separate, 100%-identical-looking setter functions for Every. Single. Field.

Look, I feel you. I too have been annoyed by various places where I see what I consider boilerplate but this is the price of admission for working with Elm. You have to deal with the tradeoffs that have been selected by the language creator. It’s not like you pay this price for nothing, you do get the niceties of Elm in return.

I, like many others, have wasted countless hours trying to get rid of boilerplate. I too have come to the conclusion that all this time would have been much better spent implementing something.

The main issue is that there is no solution. It’s not like the boilerplate could be eliminated without tradeoffs. Getting rid of some boilerplate would mean dealing with a fresh set of problems that the solution for the boilerplate raises. Most of the time these problems are not obvious to someone just wanting to get rid of the boilerplate BUT the core team of Elm analyzes the consequences of the proposed solutions and trust me, there have been countless man-hours poured in the records update discussions already. It has been going on since 2015 at least.

The best way I’ve seen for dealing with boilerplate is to isolate it. The biggest breakthrough for me was when I saw @rtfeldman’s elm-spa-example Data folder with the JSON decoders and encoders isolated in each data type module. Same goes for the Requests folder. Not having to deal with those bits of code in the page code is a huge improvement for me.

As to the kind of boilerplate you are dealing with, look into form management libraries. Maybe one of the libraries would make more sense for your case. Be aware that in choosing to go that route you will still have to deal with the tradeoffs of using those libraries. As I said, there are no solutions, only tradeoffs.

I think that the price of admission for using Elm is quite low. I still have days when my blood sugar is low or I haven’t slept well and I get cranky and frustrated by some aspect but in the good days I pay the price gladly and I go about it happily writing JSON decoders like Data scanning for life forms.

[MarkHamburg]

Where I find myself wanting setter functions most often is in constructing arguments to map functions. I want to update a field on every member of a collection. Or more often I want to update a field on the first element of a model/command tuple. Writing the function boilerplate starts to get to the point where there is enough boilerplate that it can obscure the intention because the reader has to do the level shift to think about the inline function. If one writes the function out of line, it can have a name which clarifies a lot but now one is writing setFoo for the umpteenth time. (I currently just accept Richard’s position and roll with it. It’s not like it’s that hard to write.)

That said, at least as often as I want setFoo : b -> { a | foo : b } -> { a | foo : b }, I also want asFooIn : { a | foo : b } -> b -> { a | foo : b } and I can’t imagine that writing flip setFoo all the time would do anything for clarity. So, if there were to be syntactic extension to handle this, it would need to be done with careful consideration to get the parameter order “right”.

Mark

[madasebrof]

All of us who have been developing in Elm for any amount of time has dealt with this.

Without rehashing what has been said, I’d just add that, philosophically, this is the price you pay for perfection.

That may sound grandiose, but what I mean by that is that, with Elm, you can only write production code. There is no such things as “hacking” in Elm.

If it compiles, you can ship it.

The downside is that this is part of the price you pay. You have to be specific. That’s why nothing ever breaks.

Also, I disagree with your statement about Dicts. We use LOTS of forms (multi-page) and ended up going with Dicts, but using Dicts of union types. Type-safe AND no boilerplate!

e.g. in the model:

type alias Model =
    { formFields : Dict String AnswerValue }

and the type:

type AnswerValue
    = AnswerValueNone
    | AnswerValueString String
    | AnswerValueBool Bool
    | AnswerValueCsv String
    | AnswerValueFloat Float
    | AnswerValueInt Int
    | AnswerValueRangeInt ( String, Int, String, Int )

then a single update function, something like:

UpdateFormAnswerValue key answerValue ->
    let
        updatedFields =
            Dict.insert key answerValue model.formFields
    in
        ( { model | formFields = updatedFields }, Cmd.none )

(This all rather simplified from what we actually do, as we also push data out a WebSocket as we update, we support multiple forms, etc. But hopefully, you get the gist!)

We use a dot notation to name fields. e.g. user.contact.email, user.contact.address, etc.

We have a separate semi-complex form validation library that validates things like email, phone, currency, values based on values in other fields, required fields, max/min values, etc.

Works brilliantly.

(Someday I’ll write it all up, or open source it…)

Anyway, best of luck, I HIGHLY recommend sticking with Elm!!!

[skosch]

Thanks @madasebrof.

I think being able to use Dicts in a more type-safe way would already go a long, long way in fixing this problem. I know about issue #774 but it doesn’t look like this will land anytime soon.

I’ve switched to ReasonML for now. It suffers from much of the same verbosity, and isn’t nearly as clean and lovely a language as Elm. It does, however, have human-readable (i.e. JSX) markup syntax and lightning-fast compile times – the two other things that are real pain points in Elm. Oh, and the JS bundles are tiny. I really hope Elm will catch up soon

[madasebrof]

A super talented developer I work with explored that ecosystem, for exactly the same reasons you are.

After about two weeks, he gave up. His exact quote was:

My OCaml exploration is mostly done. It’s a nice language but not as elegant as Elm, especially wrt to frontend development.

Either way, you’ll learn a lot! I admit I found a few things frustrating w/Elm at first. But for me, the pros FAR out weight the cons.

I also think a lot of the frustration w/ Elm at first is just that it feels like you have to write too much code to “see something”. The flip side is that, as your codebase grows, you’ll be stunned at how easy it is to refactor, and how bullet-proof the code is. Most codebases grow exponentially more complex as they grow in LOC whereas Elm seems to stay linear.

Anyway, best of luck.

[rtfeldman]

Worth noting that dictionaries have different use cases than records; having to handle cases where keys might not be present (when you know for a fact that they must be) is a serious drawback if you don’t actually need dynamic keys!

Shrinking bundle sizes and compile times are the two biggest improvements coming in the next major Elm release, and yeah I think there’s generally consensus that those were problems worth prioritizing.

Also worth noting that Elm already has human-readable syntax for representing HTML: plain function calls. At work our designers write plain Elm, people we’ve hired straight out of coding bootcamps write plain Elm in their first week, and when our back-end engineers need to make front-end changes, they write plain Elm. Lots of humans are happily reading plain Elm just fine, and there’s a clear reduction in complexity to having the entire code base share a single readable syntax!

That said, of course everybody’s got their own syntax preferences, and certainly ReasonML has a lovely community! I love hanging out with those folks at conferences and such. I think both Reason and Elm are both fine languages with bright futures. ️

[evancz]

Mod Note: I just removed a bunch of posts about the current and future behavior of value vs defaultValue that could be better explored in a separate thread.

[system]

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.