# Handling json data with mixed structure

**URL:** <https://discourse.elm-lang.org/t/handling-json-data-with-mixed-structure/4373>\
**Category:** Request Feedback\
**Created:** [September 25, 2019, 7:59am UTC](https://discourse.elm-lang.org/t/handling-json-data-with-mixed-structure/4373 "2019-09-25T07:59:31Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![Nikolis\_Galerakis](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.elm-lang.org/nikolis_galerakis/32/2479_2.png) [@Nikolis\_Galerakis](https://discourse.elm-lang.org/u/Nikolis_Galerakis)\
**Post date:** [September 25, 2019, 7:59am UTC](https://discourse.elm-lang.org/t/handling-json-data-with-mixed-structure/4373/1 "2019-09-25T07:59:31Z")

</div>

Hello all

I am developing an app that is expected to update it’s data continiously using websocket connections (I am only bringing that up to calrify that several messages with data could be comming per second.)

I am getting the data need to update my model from Kraken (bitcoin trading plarform) but the problem I am currently wrestling with right now is that the data are actually coming in a structure as shown below:

```
[
  34,
  {
    "a": [
      "0.042062",
      21000,
      "21000.00000000"
    ],
    "b": [
      "0.041960",
      63306,
      "63306.95000000"
    ],
    "c": [
      "0.041960",
      "0.00000011"
    ],
    "v": [
      "2453051.00336514",
      "8208496.19778969"
    ],
    "p": [
      "0.041922",
      "0.042844"
    ],
    "t": [
      606,
      1557
    ],
    "l": [
      "0.041383",
      "0.041383"
    ],
    "h": [
      "0.042414",
      "0.044987"
    ],
    "o": [
      "0.042107",
      "0.044318"
    ]
  },
  "ticker",
  "ADA\/EUR"
]

```

As you can probably guess the mostof the field are float but somehow serialized as strings  
and in some cases there is an list of string and inbitween the string there is an int as for example in the “a” field.

My question was how would you handle such a case when it comes to decode into elm values.  
Would it be a good idea to create some type alias to immitate this kind of structure (is it even possible given the mixed lists ?) or would you just go with regex and just find the values of interest and if so what would be the performance implications in your experience ?

thanks in advance

---

<div class="post-metadata">

**Author:** ![eike](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.elm-lang.org/eike/32/749_2.png) [@eike](https://discourse.elm-lang.org/u/eike)\
**Post date:** [September 25, 2019, 9:37am UTC](https://discourse.elm-lang.org/t/handling-json-data-with-mixed-structure/4373/2 "2019-09-25T09:37:48Z")

</div>

In general, the best way to proceed is to first design an Elm data type that describes the data in a way that is appropriate for your application. Then you would write a `Decoder` to turn this JSON monstrosity into your nice type. I would not try to imitate the structure of this JSON.

Unfortunately, I think the JSON example is to confusing for me to be able to give more specific advice. Maybe you can explain what the different parts of the JSON represent?

---

<div class="post-metadata">

**Author:** ![rupert](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.elm-lang.org/rupert/32/1775_2.png) [@rupert](https://discourse.elm-lang.org/u/rupert)\
**Post date:** [September 25, 2019, 9:45am UTC](https://discourse.elm-lang.org/t/handling-json-data-with-mixed-structure/4373/3 "2019-09-25T09:45:24Z")

</div>

Elm decoders won’t treat strings and numbers leniently, strings must be quoted and numbers not.  
Elm also requires that the elements in a List all have the same type.

I think you could use `oneOf` on the items in the list?

Something like:

```auto
Decode.oneOf 
    [ Decode.float
    , Decode.map (\val -> String.toFloat val |> Maybe.withDefault 0.0) Decode.string
    ]

```

Would that help?

---

<div class="post-metadata">

**Author:** ![Nikolis\_Galerakis](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.elm-lang.org/nikolis_galerakis/32/2479_2.png) [@Nikolis\_Galerakis](https://discourse.elm-lang.org/u/Nikolis_Galerakis)\
**Post date:** [September 25, 2019, 9:51am UTC](https://discourse.elm-lang.org/t/handling-json-data-with-mixed-structure/4373/4 "2019-09-25T09:51:29Z")

</div>

Thanks for taking the time to respond and thanks for calling it monstrosity because for a moment I thought something is wrong with me for considering that this api is making trivial things complex.

well as you can see the the last string on the list “ADA\ / EUR” indicates a pair of currencies.  
the rest of the structure are float and int values representing “o” opening values “a” best bid and so own.  
but in general it’s just list of numbers (despite serialized as strings).  
I am not sure if this covers your question.

---

<div class="post-metadata">

**Author:** ![Nikolis\_Galerakis](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.elm-lang.org/nikolis_galerakis/32/2479_2.png) [@Nikolis\_Galerakis](https://discourse.elm-lang.org/u/Nikolis_Galerakis)\
**Post date:** [September 25, 2019, 9:59am UTC](https://discourse.elm-lang.org/t/handling-json-data-with-mixed-structure/4373/5 "2019-09-25T09:59:13Z")

</div>

Yes that is really helpfull and most probably the way to go for me, thank you  
The documentation for the method is a bless as well 😂

> Why would someone generate JSON like this? Questions like this are not good for your health. The point is that you can use `oneOf` to handle situations like this!

If you can direct me to an example that would be even better, but you helped me enough already.

---

<div class="post-metadata">

**Author:** ![eike](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.elm-lang.org/eike/32/749_2.png) [@eike](https://discourse.elm-lang.org/u/eike)\
**Post date:** [September 25, 2019, 10:37am UTC](https://discourse.elm-lang.org/t/handling-json-data-with-mixed-structure/4373/6 "2019-09-25T10:37:46Z")

</div>

Well, for example I don’t know if there are always precicely the fields “a”, “b”, “c”, “v”, “p”, “t”, “l”, “h”, “o” or if every result has different fields. In the former case, you might want to create an Elm record with precicely those fields (maybe with more descriptive names). In the latter case you’d want to decode to a `Dict`. For each of these fields, does every number indicate the same kind of value (e.g. the development of the price) or does the list represent a tuple (e.g. (number\_of\_shares, minimum\_bid, maximum\_bid)). Depending on what it is you simply want to decode to a list (using @rupert’s snippet, for example) or you might want to post-process it to a tuple or record on the Elm side.

Also, the complete thing seems to be a list of very different kinds of objects. You almost certainly want to handle that part in the decoder so that the rest of your app doesn’t have to handle this inhomogeneous list. That is probably the most annoying part and I don’t see a particularly elegant way of doing it. I think `Decode.index` is the way to go. For example, if you introduce an Elm type

```auto
type alias KrakenData =
    { id : Int
    , prices : KrakenPrices
    , type_ : String
    , currencies : String
    }

```

(I don’t know if these names make sense) you can decode to this type with

```auto
decodeKrakenData : Decoder KrakenData
decodeKrakenData = Decode.map4 KrakenData
    (Decode.index 0 Decode.int)
    (Decode.index 1 decodePrices)
    (Decode.index 2 Decode.string)
    (Decode.index 3 Decode.string)

```

where `decodePrices` is your `Decoder` for the prices object. You might also want to replace the `Decode.String`s by custom `Decoder`s that check that the type is actually `"ticker"` and that already split the string of two currencies into a tuple (or similar).

---

<div class="post-metadata">

**Author:** ![Nikolis\_Galerakis](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.elm-lang.org/nikolis_galerakis/32/2479_2.png) [@Nikolis\_Galerakis](https://discourse.elm-lang.org/u/Nikolis_Galerakis)\
**Post date:** [September 30, 2019, 8:30am UTC](https://discourse.elm-lang.org/t/handling-json-data-with-mixed-structure/4373/7 "2019-09-30T08:30:17Z")

</div>

I end up going the way you indicated because it looks more straightforward-elegant but although the json is as follows

`[34,{"a":["0.033967",5683,"5683.06014663"],"b":["0.033910",93443,"93443.50000000"],"c":["0.033928","1594.05655185"],"v":["846219.74717121","2737731.85431621"],"p":["0.033530","0.033756"],"t":[244,709],"l":["0.033000","0.033000"],"h":["0.034091","0.034500"],"o":["0.034072","0.034459"]},"ticker","ADA/EUR"]`

I get a complain by elm:

`"Failure \"Expecting an ARRAY\" <internals>"`

my decoder looks like this:

```
dataDecoder : Decoder DataUpdate
dataDecoder =
    map4 DataUpdate
        (index 0 int)
        (index 1 decoderAssetMetaInfoUp)
        (index 2 string)
        (index 3 string)

```

any idea where this might be coming from ?

---

<div class="post-metadata">

**Author:** ![eike](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.elm-lang.org/eike/32/749_2.png) [@eike](https://discourse.elm-lang.org/u/eike)\
**Post date:** [September 30, 2019, 8:57am UTC](https://discourse.elm-lang.org/t/handling-json-data-with-mixed-structure/4373/8 "2019-09-30T08:57:03Z")

</div>

Hm. I don’t see a mistake in this code. Maybe I’m overlooking something or the failure is in `decoderAssetMetaInfoUp`? If you post that as well, I will take a look.

---

<div class="post-metadata">

**Author:** ![Nikolis\_Galerakis](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.elm-lang.org/nikolis_galerakis/32/2479_2.png) [@Nikolis\_Galerakis](https://discourse.elm-lang.org/u/Nikolis_Galerakis)\
**Post date:** [September 30, 2019, 9:25am UTC](https://discourse.elm-lang.org/t/handling-json-data-with-mixed-structure/4373/9 "2019-09-30T09:25:45Z")

</div>

This is my modeling so far ignoring the a,b field for the shake of simplicity

```
type alias DataUpdate =        
    { numb : Int               
    , metaInf : AssetMetaInfoUp
    , apiStr : String
    , pair : String            
    }

type alias AssetMetaInfoUp =
    { c : List String
    , v : List String
    , p : List String
    , t : List Int
    , l : List String
    , h : List String
    , o : List String
    }

decoderAssetMetaInfoUp : Decoder AssetMetaInfoUp
decoderAssetMetaInfoUp =
    Json.Decode.succeed AssetMetaInfoUp
    {-- |> required "a" (list string) 
    |> required "b" (list string) --}
    |> required "c" (list string)
    |> required "v" (list string)
    |> required "p" (list string)
    |> required "t" (list int)
    |> required "l" (list string)
    |> required "h" (list string)
    |> required "o" (list string)

dataDecoder : Decoder DataUpdate
dataDecoder =
    map4 DataUpdate
        (index 0 int)
        (index 1 decoderAssetMetaInfoUp)
        (index 2 string)
        (index 3 string)

```

this is the object pretty printed from the javascript side

```
[
  139,
  {
    "a": [
      "0.00028250",
      1292,
      "1292.28197076"
    ],
    "b": [
      "0.00028170",
      165,
      "165.49382420"
    ],
    "c": [
      "0.00028250",
      "51.05802924"
    ],
    "v": [
      "4307.99605359",
      "21535.20012158"
    ],
    "p": [
      "0.00027948",
      "0.00027057"
    ],
    "t": [
      93,
      214
    ],
    "l": [
      "0.00027260",
      "0.00026280"
    ],
    "h": [
      "0.00028370",
      "0.00028370"
    ],
    "o": [
      "0.00027280",
      "0.00027100"
    ]
  },
  "ticker",
  "ATOM/XBT"
]

```

This is what it looks like when I log it through Debug.toString in elm side :

`"\"[139,{\\\"a\\\":[\\\"0.00028180\\\",747,\\\"747.17000000\\\"],\\\"b\\\":[\\\"0.00028080\\\",157,\\\"157.49382420\\\"],\\\"c\\\":[\\\"0.00028080\\\",\\\"0.00000010\\\"],\\\"v\\\":[\\\"4315.99605359\\\",\\\"21543.20012158\\\"],\\\"p\\\":[\\\"0.00027948\\\",\\\"0.00027057\\\"],\\\"t\\\":[97,218],\\\"l\\\":[\\\"0.00027260\\\",\\\"0.00026280\\\"],\\\"h\\\":[\\\"0.00028370\\\",\\\"0.00028370\\\"],\\\"o\\\":[\\\"0.00027280\\\",\\\"0.00027100\\\"]},\\\"ticker\\\",\\\"ATOM/XBT\\\"]\""`

and this is the error:

`"Failure \"Expecting an ARRAY\" <internals>"`

thank you very much in advance for being so eager to help !

---

<div class="post-metadata">

**Author:** ![eike](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.elm-lang.org/eike/32/749_2.png) [@eike](https://discourse.elm-lang.org/u/eike)\
**Post date:** [September 30, 2019, 9:41am UTC](https://discourse.elm-lang.org/t/handling-json-data-with-mixed-structure/4373/10 "2019-09-30T09:41:34Z")

</div>

The decoder looks fine, but the string you’re using does not. It seems to have gone through an extra layer of string escaping. My guess is that you pass in the JSON string on the JavaScript side but your Elm app expects to receive an object. It’s probably easiest to insert a `JSON.parse` call on the JavaScript side before passing the object to your Elm app.

(To verify that that’s the problem, try inserting the pretty-printed JSON in on the Elm side as string (use `"""a triple quoted string like this"""` to allow for line-breaks in the string and to avoid having to escape the single quotes). Then call your decoder on that.)

---

<div class="post-metadata">

**Author:** ![Nikolis\_Galerakis](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.elm-lang.org/nikolis_galerakis/32/2479_2.png) [@Nikolis\_Galerakis](https://discourse.elm-lang.org/u/Nikolis_Galerakis)\
**Post date:** [September 30, 2019, 9:52am UTC](https://discourse.elm-lang.org/t/handling-json-data-with-mixed-structure/4373/11 "2019-09-30T09:52:03Z")

</div>

You were absolutely right just using JSON.parse did the Job.

I was getting the value through a onMessage call back which is reasonably a json string but somehow I had in mind that it’s coming from an API(websockets this time) so javascript automatically parses it to a Json object.

thank you

---

<div class="post-metadata">

**Author:** ![system](https://canada1.discourse-cdn.com/flex035/uploads/elm_lang/original/1X/50a05e53677a2c3b47776d7abd0f113eb50193a1.png) [@system](https://discourse.elm-lang.org/u/system)\
**Post date:** [October 10, 2019, 9:52am UTC](https://discourse.elm-lang.org/t/handling-json-data-with-mixed-structure/4373/12 "2019-10-10T09:52:05Z")

</div>

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.