Loading some things in parallel into a record

Request Feedback
1

Hi!

I’m trying to load a few things in parallel, in my case via ports (Could be some other Task, not relevant to the question since the only way to do real parallelism is with Cmd.batch AFAIK).

  • When I’m done, I’d like to store them in a record for ease of access in the application.
  • When any of the things fails, I’ll move to the error state (kind of like JS Promise.all).
type alias Thing

type alias Things =
    { thing1 : Thing
    , thing2 : Thing
    , thing3 : Thing
    }

:question: I’d love to know if there is a different or better way to do this for this use case.

Here is how I’ve set this up a couple of times (small example):

I split my model in two steps. Loading with the remaining items to load, and a Dict for the temporary results. Loaded with the record for when I’m done. To load things, I request them with Cmd.batch passing out an identifier, which I get back with maybe a response at some point.

type Model
    = Loading Int (Dict String Thing)
    | Loaded (Maybe Things)

type Msg
    = GotThing ( String, Maybe Thing )

init : () -> ( Model, Cmd Msg )
init () =
    let
        thingsToLoad =
            [ "thing1"
            , "thing2"
            , "thing3"
            ]
    in
    ( Loading (List.length thingsToLoad) Dict.empty
    , thingsToLoad |> List.map getThing |> Cmd.batch
    )

When getting things out of order, I store them on the tmp Dict, and decrement the remaining int.

update : Msg -> Model -> ( Model, Cmd Msg )
update msg model =
    case model of
        Loading remaining things ->
            case msg of
                GotThing ( name, Just thing ) ->
                    let
                        things_ =
                            Dict.insert name thing things
                    in
                    if remaining == 1 then
                        ( Loaded <| dictToThings things_, Cmd.none )

                    else
                        ( Loading (remaining - 1) things_, Cmd.none )

                GotThing ( name, Nothing ) ->
                    ( Loaded Nothing, Cmd.none )

                _ ->
                    ( model, Cmd.none )

        Loaded things ->
                ( model, Cmd.none )

When done, I transition to Loaded and parse the dict into the record:

dictToThings : Dict String Thing -> Maybe Things
dictToThings things =
    Maybe.map3
        (\t1 t2 t3 ->
            { thing1 = t1
            , thing2 = t2
            , thing3 = t3
            }
        )
        (Dict.get "thing1" things)
        (Dict.get "thing2" things)
        (Dict.get "thing3" things)

There are a few things I’m not particularly happy about, but it gets the job done, so I’m interested to know if there are other approaches, or maybe a library that eases this kind of parallel loading of things.

Here is an ellie with the example to tinker around: https://ellie-app.com/6qr9whQxFBga1

If a more concrete use case helps think about solutions, I have had to do this with loading assets for a game in parallel at the start, and with loading firebase data from JS in parallel when entering a new view.

2

If I had an unknown/dynamic number of things to load, then I would use a Dict for the loaded things too instead of a record, and I wouldn’t have the dictToThings function which is sort of error prone because of the order dependence of the code. Then on the application I would have to deal with Maybe Things instead. But that would be a different use case.

3

Not sure if this fits your situation

If we’re talking about a known set of things, and prefer to work with a record, then I might work with a adding a “setter” function to the Msg

type Msg
    = GotThing (Model -> Thing -> Model) Thing

-- `subscriptions = always (receiveThing GotThing)` has to change..

then I might invoke my Cmds with the custom setters

Cmd.batch
    [ doGetThing (\m t -> { m | thing1 = t }) "thing1"
    , doGetThing (\m t -> { m | thing2 = t }) "thing2"
    ]

{-| assuming Model is now like

    type alias Model =
        { thing1 : Maybe Thing
        , thing2 : Maybe Thing
        }
-}

and finally update them independently without checking remaining

update msg model =
    case msg of
        GotThing setThing newThing ->
            ( setThing model newThing, Cmd.none )
4

The difference is that the record fields become all Maybes and then any time you have to access the thing you have to deal with the maybeness.

5

I think I would do something like this (untested):

type alias Things =
    { thing1 : Thing
    , thing2 : Thing
    , thing3 : Thing
    }

type alias Responses =
    { thing1 : Maybe Thing -- or perhaps 'RemoteData Thing'
    , thing2 : Maybe Thing
    , thing3 : Maybe Thing
    }

type Model
    = Loading Responses
    | Loaded Things
    | Error String

type Msg
    = GotThing1 (Result String Thing)
    | GotThing2 (Result String Thing)
    | GotThing3 (Result String Thing)

init : () -> ( Model, Cmd Msg )
init () =
    ( Loading
        { thing1 = Nothing
        , thing2 = Nothing
        , thing3 = Nothing
        }
    , Cmd.batch
        [ getThing1
        , getThing2
        , getThing3
        ]
    )

update : Msg -> Model -> ( Model, Cmd Msg )
update msg model =
    case model of
        Error message ->
            ( model, Cmd.none )

        Loaded things ->
            ( model, Cmd.none )

        Loading responses ->
            case msg of
                GotThing1 (Err message) ->
                    ( Error message, Cmd.none )
        
                GotThing1 (Ok thing1) ->
                    checkIfLoaded { responses | thing1 = Just thing1 }

                GotThing2 (Err message) ->
                    ( Error message, Cmd.none )
        
                GotThing2 (Ok thing2) ->
                    checkIfLoaded { responses | thing2 = Just thing2 }

                GotThing3 (Err message) ->
                    ( Error message, Cmd.none )
        
                GotThing3 (Ok thing3) ->
                    checkIfLoaded { responses | thing3 = Just thing3 }

            
checkIfLoaded : Responses -> ( Model, Cmd Msg )
checkIfLoaded results =
    case Maybe.map3 Things results.thing1 results.thing2 results.thing3 of
        Just things ->
            ( Loaded things, Cmd.none )

        Nothing ->
            ( Loading results, Cmd.none )

This avoids the counter and the string keys, and has the advantage of extending nicely to the case where thing1, thing2 and thing3 are different types.

5 Likes
6

I like the explicitness and how it avoids the magic Int and String keys on the version I posted. Thanks @ianmackenzie.

7

The two ideas - using a Msg containing a function and having a temporary data structure with maybes - can most likely be combined, so that Result handling and updating the data structure is done once instead of n times. Though, with this example (3 fields) it’s not really necessary.

8

There is no parallelism :sunny: This is due to the JS virtual-machine in the browser.

Regarding the app - I would put all the individual things in a record and wrap them each in the RemoteData-type. Along with that I’d write a function that looks at the record and gives a bool on whether everything is loaded successfully.

9

I disagree. While JS execution is definitely single threaded, there is parallelism with workers, Image loading, HTTP requests, and now the JS realm in iframe DOM elements in some browsers.

In my case the underlying operations come from Http so they are definitely running in parallel.

10

“There is no parallelism” was meant in the context of your network-requests - I wouldn’t say async network-requests are reasonable to describe as “parallelism”… But we don’t have to agree on this! :sunny:

Did you try the RemoteData thing? I think (for most cases) it’s a lot nicer than dictionaries and custom-type-models.

11

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.