Elm/json re-implemented in Elm for learning

[lydell]

I’ve always found JSON encoders to be easier to understand than decoders. Encoders are just functions that take some Elm data type as input and return a Json.Encode.Value. Then you can give that Value to Json.Encode.encode to turn it into a JSON string.

But what are decoders? They have the type Decoder a, but what is that really?

To learn more thoroughly how decoders work, I implemented elm/json in Elm. Here’s the result:

lydell/elm-jason

The key insight I gained is: Decoders are just functions in disguise. They are functions that take a raw JSON value as input, and return a Result.

You can think of a decoder like this:

type Decoder a = Json.Decode.Value -> Result Json.Decode.Error a

Except that you can’t call those functions manually. Only Json.Decode.decodeString and Json.Decode.decodeValue can. Once I understood this, decoders didn’t feel so magical anymore.

A couple of other little things I learned:

• Json.Decode.Value and Json.Encode.Value are actually the same type. It’s defined in Json.Encode and Json.Decode exposes an alias.
• Json.Decode.oneOf [] always fails. Not because the JSON input is invalid, but because of the empty list of decoders.
• Json.Decode.index checks for too large index but not too small (issue).
• Recursive fuzzing with elm-test.

[harrysarson]

How does the speed compare to the native elm json?

[lydell]

I haven’t benchmarked. Maybe it’s time I learn how to use elm-benchmark

It does more work, so it should be slower. But how much? No idea.

[lydell]

I’ve now started experimenting what an alternate API could look like:

lydell/elm-jason

I’ve went with a pretty minimalistic design.

The big change is that a Decoder is now defined as:

type alias Decoder a = JsonValue -> Result Error a

In other words, I don’t hide the fact that decoders are functions and let you call them yourself.

JsonValue is exposed and not opaque, so you’re allowed to peek inside it.

There is no Encode module. Instead you build up a JsonValue using its value constructors and then stringify that. I’m not sure if this is a good idea or not. It might be nice to have helper functions for creating objects, and for Maybes.

I’ve added optionalField, which I like a lot. It returns a Nothing if the field is missing or set to null, because in practice I’ve found that to be the most useful. Sometimes APIs omit fields, sometimes they set them to null. Sometimes they’re not even consistent.

There is no alternative to Json.Decode.maybe, since it don’t like it. It swallows errors. And when you have optionalField there’s less use for it.

There is no alternative to Json.Decode.null either. I’ve never used it. I think it’s better to use nullable. I’ve added a withDefault decoder as well if you want to replace null/Nothing with some other value.

I’ve added andMap from Json.Decode.Extra to allow for decoding objects with more than 8 fields. Wait – 9 fields. I added a map9 to not discriminate any digits. (Sorry 0, map0 makes no sense!)

The error messages are short and to the point. They show you where in the JSON the error occurred, what the expected type is and what the actual value is. The actual value is truncated to at most 100 characters. There is also a function for just printing the type of the actual value instead, which can be useful if you’re dealing with sensitive data.

There are other little things as well. You’ll find them in the code if you are interested.

[miniBill]

Well, map0 is actually succeed/constant.

I really like that you’re experimenting with different ideas, let us know about benchmarks!

[ilias]

Cool! I did a similar-ish thing back in the 0.18 days: https://github.com/zwilias/elm-json-in-elm

Linking just so you can have a look at that code as well, who knows, you might end up getting some inspiration!

[lydell]

@miniBill Ah! Yes, that makes sense!

@ilias Wut, that is so cool! You even wrote a parser! Our implementations are really similar – it kinda felt like reading my own code when I looked it through.

Here’s another thing I learned about JSON decoders today: I’ve always used fail wrongly inside andThen.

type User
    = Registered String
    | Anonymous


userDecoder : Json.Decode.Decoder User
userDecoder =
    Json.Decode.field "tag" Json.Decode.string
        |> Json.Decode.andThen
            (\tag ->
                case tag of
                    "registered" ->
                        Json.Decode.map Registered
                            (Json.Decode.field "name" Json.Decode.string)

                    "anonymous" ->
                        Json.Decode.succeed Anonymous

                    _ ->
                        -- I’ve always added the bad value to the error message here.
                        Json.Decode.fail ("Invalid User tag: " ++ tag)
                        -- But elm/json shows the bad value anyway! This is better:
                        -- Json.Decode.fail "Invalid User tag"
            )

The error message will look like this, mentioning the bad value “admin” twice:

Problem with the value at json.tag:

    "admin"

Invalid User tag: admin