Help: Decode Json from server

Learn
1

I am struggling to decode a Json string.

I receive the following json from a server:

“[{"A":1,"B":6},{"A":2,"B":7},{"A":3,"B":8},{"A":4,"B":9},{"A":5,"B":10}]”

I have writen the following code to decode the json into my own types:

import Json.Decode as Decode exposing (Decoder, decodeString, float, int, list, string)

type alias RowRecord =
    { a : Int
    , b : Int
    }

type alias DF =
    List RowRecord

rowDecoder : Decoder RowRecord
rowDecoder =
    Decode.map2 RowRecord
        (Decode.field "A" int)
        (Decode.field "B" int)


dfDecoder : Decoder DF
dfDecoder =
    Decode.list rowDecoder

My code compiles when using elm repl:

> jsonString = "[{\"A\":1,\"B\":6},{\"A\":2,\"B\":7},{\"A\":3,\"B\":8},{\"A\":4,\"B\":9},{\"A\":5,\"B\":10}]"

> Decode.decodeString dfDecoder jsonString
Ok [{ a = 1, b = 6 },{ a = 2, b = 7 },{ a = 3, b = 8 },{ a = 4, b = 9 },{ a = 5, b = 10 }]
    : Result Decode.Error DF

However, my attempt to decode the json file when coming from a server is failing. The code is compiling but I reveive a message saying

“BadBody "Problem with the given value:\n\n\"\\\"[{\\\\\\\"A\\\\\\\":1,\\\\\\\"B\\\\\\\":6},{\\\\\\\"A\\\\\\\":2,\\\\\\\"B\\\\\\\":7},{\\\\\\\"A\\\\\\\":3,\\\\\\\"B\\\\\\\":8},{\\\\\\\"A\\\\\\\":4,\\\\\\\"B\\\\\\\":9},{\\\\\\\"A\\\\\\\":5,\\\\\\\"B\\\\\\\":10}]\\\"\"\n\nExpecting a LIST"” }

module Main exposing (main)

import Browser
import Html exposing (Html, div, input, label, text)
import Html.Events exposing (onInput)
import Http
import Json.Decode as Decode exposing (Decoder, int)



-- Define types and decoders


type alias RowRecord =
    { a : Int
    , b : Int
    }


rowDecoder : Decoder RowRecord
rowDecoder =
    Decode.map2 RowRecord
        (Decode.field "A" int)
        (Decode.field "B" int)


type alias DF =
    List RowRecord


dfDecoder : Decoder DF
dfDecoder =
    Decode.list rowDecoder


getDataFrame : Cmd Msg
getDataFrame =
    Http.get
        { url = "http://localhost:5051/df"
        , expect = Http.expectJson DFReceived dfDecoder
        }



-- define Model


type alias Model =
    { df : DF, error : String }


initialModel : Model
initialModel =
    { df = [ RowRecord 0 0 ], error = "" }



-- define Msg


type Msg
    = InputChanged String
    | DFReceived (Result Http.Error DF)


update : Msg -> Model -> ( Model, Cmd Msg )
update msg model =
    case msg of
        InputChanged newInput ->
            ( model, getDataFrame )

        DFReceived result ->
            case result of
                Ok newDF ->
                    ( { model | df = newDF }, Cmd.none )

                Err error ->
                    ( { model | error = Debug.toString error }, Cmd.none )


view : Model -> Html Msg
view model =
    div []
        [ label [] [ text "Label" ]
        , input [ onInput InputChanged ] []
        , text (Debug.toString model)
        ]


main : Program () Model Msg
main =
    Browser.element
        { init = \flags -> ( initialModel, Cmd.none )
        , view = view
        , update = update
        , subscriptions = \_ -> Sub.none
        }

I would be greatful for any hint!

Sorry for asking these basic questions but it is really hard to find answers anywhere. This is clearly a downside of Elm. On the upside the Elm community is really helpful (also with complete newbies). Thanks for this!

2

That error message seems to have quite many escapes - are you sure the JSON from server is correct and e.g. not using some kind of double-encoding? Go to http://localhost:5051/df in browser to see what exactly you are getting from server.

3

This is what I just got from http://localhost:5051/df

“[{"A":0,"B":8},{"A":1,"B":6},{"A":2,"B":6},{"A":3,"B":6},{"A":4,"B":1},{"A":5,"B":4},{"A":6,"B":4},{"A":7,"B":5},{"A":8,"B":8},{"A":9,"B":1}]”

And if I go to http://localhost:8000 where elm reactor is running and I check for the network console I see the following:

“[{"A":0,"B":4},{"A":1,"B":2},{"A":2,"B":0},{"A":3,"B":1},{"A":4,"B":2},{"A":5,"B":2},{"A":6,"B":0},{"A":7,"B":0},{"A":8,"B":7},{"A":9,"B":3}]”

4

That is not valid JSON array. It has extra quote-characters at begin and end, and also it uses wrong quote-characters for strings.

EDIT: Actually it looks like this stupid Discourse messes with quote-characters in posts, so I’m not sure what exactly you have there - but at least those quote characters at beginning/end should not be there.

5

i.e. your elm repl test used this which works

> jsonString = "[{\"A\":1,\"B\":6},{\"A\":2,\"B\":7},{\"A\":3,\"B\":8},{\"A\":4,\"B\":9},{\"A\":5,\"B\":10}]"

but your server sends this which is wrong as it has two extra quote-characters

> jsonString = "\"[{\"A\":1,\"B\":6},{\"A\":2,\"B\":7},{\"A\":3,\"B\":8},{\"A\":4,\"B\":9},{\"A\":5,\"B\":10}]\""
1 Like
6

Puh. You are right. I was thrown off by the error message at first but thought it is the formatting of elm reactor or something. It is actually my first time dealing with json format and in python / pandas df.to_json(orient="records") apparently does not yield valid json. I fixed it using dicts. Thanks!!

7

Looking more closely at error message, I think server is actually sending

> jsonString = "\"[{\\\"A\\\":1,..."

Now according to df.to_json documentation it does create correct JSON, so it looks like server is re-encoding the output of df.to_json so that it’s surrounded by quotes and escaped.

For example using something like json.dumps(df.to_json(...)) is wrong as df.to_json already gives JSON so you can just print it as-is and don’t re-encode it again as JSON.

8

I have tried both .to_json(orient="records") and json.dumps(df.to_json(..)) both of which seem to yield invalid json. However, this seems to be a problem that the python community has not encountered. So it must be something on my end.

I fixed it converting the df manually to json-format. It works now.

Thanks again for your help!

1 Like
9

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.