Retrying failed requests... without the headache

[1hko]

I needed a generic way to retry requests that fail. I did not want to litter my model with extra state or clutter my update function by having to handle additional cases for each request’s code paths.

I found panosoft/elm-cmd-retry but it depends on native code and meets neither of the requirements set above.

Wishful thinking

It should be intuitive to use

decoder
    |> Http.get url
    |> Retry.retry 
    |> Task.attempt DataReceived

It should allow us to retry with configuration

decoder
    |> Http.get url
    |> Retry.retryWith { retries = 3, delay = 10 * Time.second }
    |> Task.attempt DataReceived

Implementation

Here’s my best attempt at the Retry module. It compiles and works as intended in my program.

module Retry exposing (..)

import Http
import Process
import Task exposing (Task)
import Time exposing (Time)


type alias Config =
    { retries : Int
    , delay : Time
    }


default : Config
default =
    { retries = 5
    , delay = 5 * Time.second
    }


retry : Http.Request a -> Task Http.Error a
retry =
    retryWith default


retryWith : Config -> Http.Request a -> Task Http.Error a
retryWith config req =
    let
        onError e =
            let
                _ =
                    Debug.log ("retrying " ++ (toString req)) e
            in
                if config.retries == 0 then
                    Task.fail e
                else
                    Process.sleep config.delay
                        |> Task.andThen (\_ -> retryWith { config | retries = config.retries - 1 } req)
    in
        req
            |> Http.toTask
            |> Task.onError onError

How to use it

As demonstrated in the example below we meet our goals. Our model is not required to keep track of extra state and it’s easy to convert any single-attempt request into a request that retries itself.

module SomeModule exposing (..)

import Retry
import User exposing (User)

...

type alias Model =
    { users : List User }

type Msg
    = UsersReceived (Result Http.Error (List User))

init : ( Model, Cmd Msg )
init =
    ( { users = [] }, getUsers )

getUsers : Cmd Msg
getUsers =
    (Decode.list User.decode)
        |> Http.get "/users.json"
        |> Retry.retry
        |> Task.attempt UsersReceived


update : Msg -> Model -> ( Model, Cmd Msg )
update msg model =
    case msg of
        UsersReceived (Ok users) ->
            { model | users = users }
                ! []

        UsersReceived (Err e) ->
            let
                _ =
                    Debug.log "request failed" e
            in
                ( model, Cmd.none )

---

Feedback

I’m new to Elm and I’m asking the community for general feedback on this module.

• Can you see problems I may encounter with it?
• Can you see ways in which it can be improved?
• Am I breaking any Elm conventions?

Possible improvements?

It would be nice to keep the Task abstraction out of the programmer’s mind entirely. Task.attempt wraps Http.send but it’s not nearly as readable.

-- [Task.attempt] leaks Task abstraction
decoder
    |> Http.get url
    |> Retry.retry
    |> Task.attempt DataReceived

-- maybe this would be better?
decoder
    |> Http.get url
    |> Retry.retry DataReceived

-- or with configuration
decoder
    |> Http.get url
    |> Retry.retryWith { ... } DataReceived

Or maybe it would be best if we continued with the Http.send pattern. However, I’m not sure how to make Retry.retry return the result that Http.send is expecting.

-- ideal api does not require hard thinking
decoder
    |> Http.get url
    |> Retry.retryWith { ... }
    |> Http.send DataReceived

---

Revision 2

Thanks to a comment from rasputin303 the API now feels more natural.

-- revision 2 api
decoder
    |> Http.get url
    |> Retry.send DataReceived

… and with configuration

decoder
    |> Http.get url
    |> Retry.sendWith { retries = 3, delay = 10 * Time.second } DataReceived

Implementation

Other formatting also increases readability.

module Retry exposing (Config, send, sendWith)

import Http
import Process
import Task exposing (Task)
import Time exposing (Time)


type alias Config =
    { retries : Int
    , interval : Time
    }


default : Config
default =
    { retries = 5
    , interval = 1 * Time.second
    }


send : (Result Http.Error a -> msg) -> Http.Request a -> Cmd msg
send =
    sendWith default


sendWith : Config -> (Result Http.Error a -> msg) -> Http.Request a -> Cmd msg
sendWith config resultToMessage req =
    req
        |> Http.toTask
        |> Task.onError (retry config req)
        |> Task.attempt resultToMessage


retry : Config -> Http.Request a -> Http.Error -> Task Http.Error a
retry config req e =
    case config.retries of
        0 ->
            let
                _ =
                    Debug.log ("retrying " ++ (toString req)) e
            in
                Task.fail e

        _ ->
            let
                _ =
                    Debug.log ("retrying " ++ (toString req)) e

                wait =
                    Process.sleep config.interval

                next =
                    req
                        |> Http.toTask
                        |> Task.onError (retry { config | retries = config.retries - 1 } req)
            in
                wait |> Task.andThen (always next)

---

Revision 3

Discourse did not allow me to add revision 3 to this post. You can see inline with the comments below.

[edkelly303]

This seems neat!

I seem to remember reading somewhere that there’s a convention not to use names like Retry.retry in libraries. Is there ever an occasion where you might want to use Retry.retry and then do something other than Http.send as a next step? If not, then maybe it’s simpler to bundle up the Http.send into your library and provide functions like Retry.send and Retry.sendWith?

[1hko]

I like it. Retry.send and Retry.sendWith fix both problems at once. Even better now the programmer can naturally understand this as a function that returns a Cmd, like Http.send does. Task abstraction out-of-sight, out-of-mind! I added a revision 2 to the post.

[Qqwy]

I wonder… would it be possible to make it more general by making Retry work with Results?

[ericgj]

Not sure it makes sense to retry a pure function that returns a Result… it’s always going to return the same Result given the same input. But I can see writing a generic retry that works with any Task. This is kind of diametrically opposed to the API that @1hko is interested in, but something like this could work:

retry : Config -> Task error success -> Task error success
retry config task =
    task |> Task.onError (retryHelp config task)


retryHelp : Config -> Task error success -> error -> Task error success
retryHelp config task error =
    if config.retries == 0 then
        let
            _ =
                Debug.log "failed retrying" error
        in
            Task.fail error
    else
        let
            _ =
                Debug.log ("retrying " ++ (toString config.retries)) error

            next =
                task
                    |> Task.onError (retryHelp { config | retries = config.retries - 1 } task)
        in
            Process.sleep config.interval
                |> Task.andThen (always next)

[1hko]

This is kind of diametrically opposed to the API that @1hko is interested in

I think this generalization is an improvement! I’m suggesting rename of retryHelp to onError as I think it makes the complete module read nicer. Hmm

Given your two implementations above:

retry : Config -> Task x a -> Task x a
onError : Config -> Task x a -> x -> Task x a

We can rewrite Retry.sendWith

sendWith : Config -> (Result Http.Error a -> msg) -> Http.Request a -> Cmd msg
sendWith config resultToMessage req =
    let
        task =
            Http.toTask req
    in
        task
            |> Task.onError (onError config task)
            |> Task.attempt resultToMessage

Now we have a more generic Retry module that can retry tasks. And in a more specialized case, it can retry Http requests. We can expose the generic too

---

Revision 3

@ericgj simplifies the retry abstraction and builds a more generic api, making the entire module more useful.

-- module Retry
default : Config
retry : Task x a -> Task x a
retryWith : Config -> Task x a -> Task x a
send : (Result Http.Error a -> msg) -> Http.Request a -> Cmd msg
sendWith : Config -> (Result Http.Error a -> msg) -> Http.Request a -> Cmd msg

Implementation

-- revision 3
module Retry exposing (Config, default, retry, retryWith, send, sendWith)

import Http
import Process
import Task exposing (Task)
import Time exposing (Time)


type alias Config =
    { retries : Int
    , interval : Time
    }


default : Config
default =
    { retries = 5
    , interval = 1 * Time.second
    }


retry : Task x a -> Task x a
retry =
    retryWith default


retryWith : Config -> Task x a -> Task x a
retryWith config task =
    task |> Task.onError (onError config task)


send : (Result Http.Error a -> msg) -> Http.Request a -> Cmd msg
send =
    sendWith default


sendWith : Config -> (Result Http.Error a -> msg) -> Http.Request a -> Cmd msg
sendWith config resultToMessage req =
    let
        task =
            Http.toTask req
    in
        task
            |> Task.onError (onError config task)
            |> Task.attempt resultToMessage


onError : Config -> Task x a -> x -> Task x a
onError config task error =
    if config.retries == 0 then
        let
            _ =
                Debug.log "failed retrying" error
        in
            Task.fail error
    else
        let
            _ =
                Debug.log ("retrying " ++ (toString config.retries)) error

            next =
                task
                    |> Task.onError (onError { config | retries = config.retries - 1 } task)
        in
            Process.sleep config.interval
                |> Task.andThen (always next)

Probably time to learn how to publish packages…