Limit on decode pipeline entries

[michaeljones]

I have been using a big union type in my approach to internationalisation. I have one entry per string that I need to translate. I also built a decoder which can decode a json file with the translated strings. The type & decoder are generated automatically from a yaml file of translations.

Unfortunately, I’ve just reached over 256 translation strings and the Decode.Pipeline based decoding is being compiled to code that uses nested functions and has reached the limit that Firefox is happy to deal with. Chrome & Edge seem to be fine. Firefox gives an InternalError: function nested too deeply error.

I realise that some i18n approaches have gone with a function per translation rather than a union type entry per translation and I guess I’ll be moving towards that. This union entry approach felt promising prior to this moment and was seemed preferable as I would prefer storing union entries as data in the model if that was needed, rather than functions, but in truth it hasn’t turned out to be necessary.

Sharing this in the hope that it helps others or that someone might be able to offer an easier solution that the re-write I’m currently facing

Edit: Title changed from limit on Union type entries as really it has nothing to do with that and more to do with the decoding I was choosing to do.

[joelq]

Elm curries functions so an n-argument function is actually n nested 1-argument functions. It sounds like Firefox doesn’t allow nesting functions deeper than 256 levels deep? Which means that in Firefox:

1. Elm functions with more than 255 arguments will break
2. You can’t type alias a record with more than 255 fields (because Elm auto-generates a constructor function for you)

I’m guessing that’s the actual problem you’re facing here and not the decoder itself?

Assuming that’s actually the problem, you could probably solve it by breaking up bigger records into smaller sub-records such that no record has more than 255 fields?

[rob]

Besides a function per translation, how about a Dict of the translations? That should be a straightforward decode from a representation like {"stringid": "translation", ...} or {"original": "translation"}. You could then use Dict.get to define a function translate lang id = ....

If you wanted more type-safety than a plain string to string lookup, you could make a big enum-like sum type of IDs for all your translated strings. I expect that shouldn’t run into this nesting limit.

[michaeljones]

Thanks @joelq - sounds like you have a better idea than me. Potentially the type alias for the record then as I did break out a type into an alias at more or less the same time so it might be that.

I considered breaking it down into smaller records as an option but it didn’t seem like the easiest transformation to make. I’ve split it all into individual functions now with a dictionary underneath and got it back to compiling. I do have a record with all the entries for a utility function so I’ll see how that goes and avoid breaking the type out into an alias I guess!

[michaeljones]

Thanks for the thought @rob - I definitely was trying for something better typed that a string key on a dictionary. That was the motivation for the path I took, but I suspect it wasn’t the optimal one. I had a union type to describe all the options and help with keeping it type safe but I then created a utility function & lookup so I could do some indirect lookups using just .field accessor functions. Based on @joelq’s comment I suspect it all fell apart when I created a type alias for part of that. Something to keep an eye on!

[rob]

Curiously, this is the second time in a week that I see a potential use for a “full dictionary” like datastructure on a finite key type. I.e.,

type LocalizedMessage
    = Welcome
    | ChangePassword
    | ...

-- generated string lookup
locFromString : String -> Maybe LocalizedMessage
locFromString s = case  s of
    "welcome" -> Just Welcome
    ...
    _ -> Nothing

-- dictionary with keys from an "enum" (sum type with only constant constructors)
type FullDict enum v = ...

-- lookup is guaranteed to succeed
get : FullDict enum v -> enum -> v

fromList : List (enum, v) -> Maybe (FullDict enum v)

type alias Localization = FullDict LocalizedMessage String

Then your localization decoder would use locFromString to decode the keys (and could fail if there were any typos), and then build a dictionary using fromList, which would fail if there were any missing messages.