RangeError: Maximum call stack size exceeded when decoding a long list

[oaalto]

I ran into the error when decoding a json value which contained a list that had over 4000 elements. I found old posts and github issues about similar situations, but none had concrete steps to fix it other than to update to the latest version of elm. I did try running the app without the debugger and it did work then, but that’s not really a solution as the debugger is pretty useful at times.

My elm version is 0.19.1-3 from npm.

My code is below.

import Json.Decode as Decode
import Json.Decode.Pipeline as Pipeline
import List.Extra as ListExtra

type Enums
= Enums (List Data)

type alias Data =
{ enumType : Type
, id : String
, name : String
}

type Type
= Application
| Operation
| IdType
| PayloadType
| MetadataName

enumsDecoder : Decode.Decoder Enums
enumsDecoder =
Decode.list dataDecoder |> Decode.andThen toEnumsDecoder

toEnumsDecoder : List Data -> Decode.Decoder Enums
toEnumsDecoder data =
Decode.succeed (Enums data)

dataDecoder : Decode.Decoder Data
dataDecoder =
Decode.succeed Data
|> Pipeline.required “type” (Decode.string |> Decode.andThen toTypeDecoder)
|> Pipeline.required “_id” Decode.string
|> Pipeline.required “name” Decode.string

toTypeDecoder : String -> Decode.Decoder Type
toTypeDecoder enumType =
case enumType of
“application” ->
Decode.succeed Application

    "operation" ->
        Decode.succeed Operation

    "idType" ->
        Decode.succeed IdType

    "payloadType" ->
        Decode.succeed PayloadType

    "metadataName" ->
        Decode.succeed MetadataName

    somethingElse ->
        Decode.fail <| "Unknown type: " ++ somethingElse

Test data can be found here: https://pastebin.com/zCDUVAF1

Am I doing something wrong or is it just the debug build that’s not handling it correctly?

[catz]

Yes. Experiencing the same on 0.19.1 & elm/browser 1.0.2. without --debug works fine.

Seems related issue github.com/elm/browser/issues/90

[allanderek]

I have no idea if this will fix your issue, but you are making a somewhat unnecessary use of andThen:

@oaalto

enumsDecoder : Decode.Decoder Enums
enumsDecoder =
    Decode.list dataDecoder |> Decode.andThen toEnumsDecoder

toEnumsDecoder : List Data -> Decode.Decoder Enums
toEnumsDecoder data =
    Decode.succeed (Enums data)

This could just be:

enumsDecoder : Decode.Decoder Enums
enumsDecoder =
    Decode.list dataDecoder |> Decode.map Enums

To me that is clearer code anyway, even if it doesn’t fix your stack problem.

[oaalto]

Thanks, unfortunately it didn’t help with the stack issue. But at least the code is clearer now.

[oaalto]

Just saw the new release of elm/browser and tested it. Unfortunately it didn’t help with my problem.

[jreusch]

The problem is in this file:

elm/browser/blob/1.0.2/src/Debugger/Expando.elm#L154

This file describes the generic model description used in the debugger. It turns your statically typed model into a “dynamically typed” Expando, which is the data structure used to render the model on the right side (including open/closed states).

Because it needs to do the Cons (::) after it calls mergeListHelp, this call is not tail-recursive. It’s esentially a map iterating 2 lists at once, which itself is essentially a right fold. Except that unlike List.map and List.foldr, this function does not include special handling of deep recursive structures.

Because the Debugger is only loaded when enabled (duh!), it works without it.

PS: The Elm Debugger is just another (special) Elm App. I think that’s pretty awesome!

[rupert]

@jreusch

Because it needs to do the Cons ( :: ) after it calls mergeListHelp , this call is not tail-recursive.

Should be fixable to tail recursive - usually takes me a while to remember how to do it, but I’ll be back with a tail recursive version unless someone beats me to it.

[TSFoster]

This would be tail recursive:

mergeListHelp : List Expando -> List Expando -> List Expando -> List Expando
mergeListHelp acc olds news =
  case (olds, news) of
    ([], _) ->
      List.reverse acc ++ news

    (_, []) ->
      List.reverse acc

    (x :: xs, y :: ys) ->
       mergeListHelp (mergeHelp x y :: acc) xs ys

And then modify the two places mergeListHelp is called from mergeListHelp oldValues newValues to mergeListHelp [] oldValues newValues.

P.S. In the second case of the original code, news is always [].

[mburri]

I encountered the same issue (Maximum call stack size exceeded)-
elm 0.19.1-3, elm/browser 1.0.2 and elm/browser 1.0.1
With a simple application that holds a List with more than 4000 entries. The debugger just crashes as soon as a message is triggered, even if the update function just returns the same model -
But this happens only in Chrome/ Chromium!
Everything works fine with Firefox or elm 0.19