Handling dependent multiple Http requests with a bad api structure

kuroski · October 18, 2018, 1:59am

Hello guys

I am currently studying elm and I came across an interesting problem…
I’m creating a pokedex using the pokeapi
The fact is that the api modeling is very verbose and complex

The scenario I want is the following:

Search a pokemon by its name
Then list its information (name, picture, description)
And list its evolutions

The cenario is:

First I have to fetch the pokemon
- https://pokeapi.co/api/v2/pokemon/inser-your-pokemon/
Then with the response, I must get its specie (this is another request) and I get the url by accessing this property pokemon.species.url
Then I must request:
- /api/v2/pokemon-species/4/ (charmander specie page)
THEN I must request its evolution chain to get all charmander evolutions and so on…

I currently came up with the below solution and I would like to know if there is a better way to do this.

I want you to focus on the request line in SearchPokemon update

-- Api module

pokeDecoder : Decoder Pokemon
pokeDecoder =
    Decode.succeed Pokemon
        |> required "name" string
        |> requiredAt [ "species", "url" ] string

specieDecoder : Decoder Specie
specieDecoder =
    Decode.succeed Specie
        |> requiredAt [ "genera", "2", "genus" ] string
        |> requiredAt [ "evolution_chain", "url" ] string

getPokemon : String -> Task Http.Error Pokemon
getPokemon term =
    let
        url =
            "https://pokeapi.co/api/v2/pokemon/" ++ term ++ "/"
    in
    Http.get url pokeDecoder
        |> Http.toTask



getSpecie : Pokemon -> Task Http.Error Specie
getSpecie pokemon =
    Http.get pokemon.specieUrl specieDecoder
        |> Http.toTask

-- Update module

type Msg
    = SearchPokemon
    | PokeSpecieResponse (Result Http.Error ( Pokemon, Specie ))

update : Msg -> Model -> ( Model, Cmd Msg )
update msg model =
    case msg of
        SearchPokemon ->
            let
                request =
                    getPokemon (String.toLower model.searchInput)
            in
            ( { model | loading = True }
            -- Here is the tricky part
            , request
                |> Task.andThen
                    (\pokemon ->
                        Task.map (\specie -> ( pokemon, specie )) (getSpecie pokemon)
                    )
                |> Task.attempt PokeSpecieResponse
            )

        PokeSpecieResponse (Ok ( pokemon, specie )) ->
            ( { model
                | loading = False
                , pokemon = Just pokemon
                , specie = Just specie
              }
            , Cmd.none
            )

        PokeSpecieResponse (Err error) ->
            ( { model | loading = False, pokemon = Nothing }, Cmd.none )

I’m omitting some of the complexity, but if for example I want to also search the evolutions, the request is gonna grow by something like this:

request
    |> Task.andThen
        (\pokemon ->
            Task.map (\specie -> ( pokemon, specie )) (getSpecie pokemon)
        )
    |> Task.andThen
        (\( pokemon, specie ) ->
            Task.map (\evolution -> ( pokemon, specie, evolution )) (getEvolutionChain specie)
        )
    |> Task.attempt PokeSpecieResponse

My problem is:

I need to request pokemon, species and evolutions
So it’s 3 requests that are dependent, if one fail I want the whole process to fail
Every task chain, I must keep the latest request result value
So at the end I pass a command to update the model

This code works, but for me to delve into the evolutions this gets even more compelling, since pokeapi returns a grotesque json and I would have to deal, and this code tends to be a mess.

And that’s it >_<, I’m doing this for fun, but this case may appear in the real world

Thaaanks a lot for your attention =D

Maxim-Filimonov · October 18, 2018, 10:53am

I’ve dealt with a similar problem on a project I’ve been working on.

After some fun with Elm and multi-chain requests. I’m leaning towards a more robust graphQL + caching layer solution.

kuroski · October 19, 2018, 10:00am

Thaaanks man, I will check your repository and try to reproduce in my structure =D
And yeah, maybe I’ll change to and api that uses graphql
thaaanks

kuroski · October 19, 2018, 12:20pm

Check it out, I’ve used your project idea and the flow now is waaay easier =DD

I’m using now the RemoteData package to help me dealing with state of multiple types

Check out the project here: https://github.com/kuroski/Pokelmon

Now I’m only dealing with commands instead of Task chain (only in this case)

update : Msg -> Model -> ( Model, Cmd Msg )
update msg model =
    case msg of
        SetSearchInput value ->
            ( { model | searchInput = value }, Cmd.none )

        SearchPokemon ->
            let
                request =
                    getPokemon (String.toLower model.searchInput)
            in
            ( { model | pokemon = Loading, specie = NotAsked }
            , Http.send PokemonLoaded (getPokemon (String.toLower model.searchInput))
            )

        PokemonLoaded (Ok pokemon) ->
            ( { model | pokemon = Success pokemon, specie = Loading }
            , Http.send SpecieLoaded (getSpecie pokemon)
            )

        PokemonLoaded (Err error) ->
            ( { model | pokemon = Failure error }, Cmd.none )

        SpecieLoaded (Ok specie) ->
            ( { model | specie = Success specie }, Cmd.none )

        SpecieLoaded (Err error) ->
( { model | specie = Failure error }, Cmd.none )

Waay simpler

joelq · October 19, 2018, 6:12pm

If I understand your original problem correctly, it’s that those two (soon to be 3…) were painful to chain together. In particular, because the data returned from each task were not just intermediate results but instead were needed to construct the final value.

If you just wanted the evolution chain, the code would look nice:

request
  |> Task.andThen getSpecie
  |> Task.andThen getEvolutionChain

but you want all the intermediate values too. This is a fairly common problem. Here are some solutions people use

Nested `andThen`

The classic solution is to nest andThens:

request
  |> Task.andThen (\pokemon ->
    getSpecie pokemon
      |> Task.andThen (\specie ->
        getEvolutionChain specie
          |> Task.andthen (\chain -> Task.succeed (pokemon, species, chain))
      )
    )

While this is nicer than having to map at every step, all those lambdas can be a bit clunky.

Flipped `andThen`

We can flatten things and get rid of parens by reorganizing things a bit. If we flip andThen's args, it will play nicer with the reverse pipe we want to use:

flippedAndThen : Task x a -> (a -> Task x b) -> Task x b
flippedAndThen value function =
  Task.andThen function value

allTogether : Task Http.Error (Pokemon, Specie, EvolutionChain)
allTogether =
    flippedAndThen request <| \pokemon ->
    flippedAndThen (getSpecie pokemon) <| \specie ->
    flippedAndThen (getEvolutionChain specie) <| \chain ->
    Task.succeed (pokemon, specie, chain)

There is currently a proposal to change pipeline decoders to use this style. Interestingly, Haskell provides syntactic sugar called do notation that is built on this approach.

Helper Function

If you run into situations like this often, it may be helpful to define a helpers like:

chain3 : (a -> b -> c -> d) -> Task x a -> (a -> Task x b) -> (b -> Task x c) -> Task x d
chain3 function taskA taskB taskC =
  taskA |> Task.andThen (\a ->
      taskB a |> Task.andThen (\b ->
        taskC b |> Task.andThen (\c -> Task.succeed <| function a b c)
        )
      )

This is just an abstraction of the nested andThen approach discussed earlier. If you use a it a lot it can be worth it since it gives you nicer invocations like:

 chain3 (\a b c -> (a, b, c))
  request
  getSpecie
  getEvolutionChain

kuroski · October 19, 2018, 6:52pm

Wow, thaaanks a LOT man, this is a very simple way of dealing with my problem.
In my recent cenario, I used Commands, but I love the Flipped andThen
Since I’m new to this kind of stuff, is mindblowing huauhauhauhuha

I’ll refactor my code to use this approach >_<

kuroski · October 19, 2018, 9:26pm

Man, the solution is beautiful huauhahuauha
I have a lot to learn >_<

SearchPokemon ->
            let
                request =
                    getPokemon (String.toLower model.searchInput)
            in
            ( { model | pokemon = Loading, specie = NotAsked }
            , Task.attempt PokemonLoaded
                <| flippedAndThen request
                <| \pokemon -> flippedAndThen (getSpecie pokemon)
                <| \specie -> Task.succeed (pokemon, specie)

Works like a charm, soon I’ll be commiting

kuroski · October 19, 2018, 9:59pm

github.com

kuroski/Pokelmon/blob/master/src/Update.elm

module Update exposing (Msg(..), update)

import Api exposing (getEvolutionChain, getPokemon, getSpecie)
import Http
import RemoteData exposing (RemoteData(..))
import Model exposing (Model, FullPokemon, Pokemon, Specie)
import Task exposing (Task)


type Msg
    = SetSearchInput String
    | SearchPokemon
    | PokemonLoaded (Result Http.Error FullPokemon)

flippedAndThen : Task x a -> (a -> Task x b) -> Task x b
flippedAndThen value function =
    Task.andThen function value


update : Msg -> Model -> ( Model, Cmd Msg )

This file has been truncated. show original

Done =D
Thanks a lot
I understant what this solution does, but I’m not fully understand how it works, soo I’m figuring out

kuroski · October 23, 2018, 11:49am

Now I fully understand the solution, this is a very beautiful and smart approach xD, me and my colleague love’d to study how this works.

Thaaanks a LOT for the lesson @joelq

I’m posting here our approach to understand it, just for recording

-- first, remove all <|
Task.attempt PokemonLoaded
    (flippedAndThen request
        (\pokemon ->
            flippedAndThen (getSpecie pokemon)
                (\specie -> Task.succeed (FullPokemon pokemon specie))
        )
    )

-- now, replace with Task.andThen
Task.attempt PokemonLoaded
    (Task.andThen
         (\pokemon ->
             Task.andThen
                 (\specie -> Task.succeed (FullPokemon pokemon specie))
                      (getSpecie pokemon)
                 )
         request
    )

{- then, try to replace all functions with <| or |>
   at this moment, this is the best we can do
   We can't remove any more "()" because of the Task.andThen notation
-} 
Task.attempt PokemonLoaded
    <| Task.andThen
         (\pokemon ->
             Task.andThen
                 (\specie -> Task.succeed (FullPokemon pokemon specie))
                      (getSpecie pokemon)
                 )
       request

joelq · October 23, 2018, 2:30pm

I love the way you slowly made a bunch of small transformations to build up an understanding of the code! Thanks for sharing each step with the comments!

system · November 2, 2018, 2:40pm

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Best way to design a web app which status depend on the success of several http requests? Learn	12	1109	August 15, 2020
Is it possible Elm reports a BadStatus error as Network error for HTTP requests taking a long to complete? Learn	2	445	October 20, 2022
JavaScript can do an http request with closure, but how does that work in Elm? Learn	6	1232	August 6, 2018
HTTP POST bad request Learn	4	905	September 1, 2020
Package: duncanmalashock/json-rest-api Show and Tell	0	664	February 4, 2018

Handling dependent multiple Http requests with a bad api structure

Nested andThen

Flipped andThen

Helper Function

Related topics

Nested `andThen`

Flipped `andThen`