RealWorld example app architected with the Effect pattern

Show and Tell
1

logo
logo1319Ă—215

A RealWorld example app version to demonstrate how to structure an Elm application using the Effect pattern.

The application is initially based on the great rtfeldman/elm-spa-example.

Introduction

Most of complex Elm application architecture examples out there use some variant of the pattern used in rtfeldman/elm-spa-example or the OutMsg pattern to have a updatable global state available in each subpage (a Session for example). The former passes the data from subpage to subpage during their initialization, so each subpage can modify it locally, the later returns an additional message from their update function handled in the top level one.

My favorite pattern to do this (and more), the Effect pattern, is most often described only in the context of tests and Cmd, for example by avh4/elm-program-test, but it can be extended to state changes and offers a clean and versatile way to handle different use cases of complex applications.

Description

The extended Effect pattern used in this application consists in definining an Effect custom type that can represent all the effects that init and update functions want to produce.

These effects can represent:

  • a Cmd value
  • a request to change the state at an upper level (for example an URL change from a subpage without an anchor)

Having a way to request an upper level state change, with or without an associated actual Cmd msg, is what lets this extended effect pattern combine the benefits of the OutMsg one with the ones from the usual effect pattern used for tests.

The init and update functions are changed to return a (Model, Effect msg), using a custom application, that will turn the Effect info actual effects through a perform function.

There are several benefits to this approach that makes it a valuable pattern for complex applications, including:

  1. All the effects are defined in a single Effect module, which acts as an internal API for the whole application that is guaranteed to list every possible effect.

  2. Effects can be inspected and tested, not like Cmd values. This allows to test all the application effects, including simulated HTTP requests.

  3. Effects can represent a modification of top level model data, like the Session when logging in, or the current page when an URL change is wanted by a subpage update function.

  4. All the update functions keep a clean and concise Msg -> Model -> ( Model, Effect Msg ) signature.

  5. Because Effect values carry the minimum information required, some parameters like the Browser.Navigation.key are needed only in the effects perform function, which frees the developer from passing them to functions all over the application.

  6. A single NoOp or Ignored String can be used for the whole application.

Differences with elm-spa-example

To focus on the architecture changes and to offer a different perspective, the elm-spa-example has been quite heavily reworked with the following changes:

  • Refactored to use an Effect type and application
  • Upgraded dependencies, including elm/http to 2.0
  • Removed client-side fields validation to focus subpage code on effects
  • Put all specific styled view code into a single View module to declutter subpages code
  • Replaced slow loading detection by a simple CSS transition
  • Refactored according to personal preferences
  • Removed custom assets

Links

Credits

Refactoring the elm-spa-example from Richard Feldman made me realize how much work went into it.
I would most likely never had released this version without such a solid base, so thank you very much @rtfeldman.

43 Likes
2

Interesting stuff, thanks for sharing.
One question: You mentioned better testability of the Effect type compared to Cmd s. I noticed that some Effects have a function or message constructor as their attached values. Does testing them work ok? I thought comparing function equality could be a bit of a problem here. I assume it relied on JS === so it wouldn’t be that much of a problem for message constructors I guess. What’s your experience with that?

2 Likes
3

I think it’s fine because these partially applied functions are either ignored or eventually fully applied during the tests.

For example let’s take FetchAuthor defined as:

type Effect msg
  = FetchAuthor (Result Errors Author -> msg) (Session -> Api.Request Author)
  | ...
  1. The first function which is a message constructor would be only used if the whole update function was simulated in tests, in which case it would be fully applied.
  2. The second uses a Session parameter only for convenience, to defer the need of a session until perform is run (the Effect is turned into a Cmd). In a test, a Session can be provided.

This effect is generated by calling Effect.fetchAuthor:

fetchAuthor : (Result Errors Author -> msg) -> Username -> Effect msg

Here is a working simple example of a test for this effect:

module HttpTests exposing (all)

import Api.Endpoint
import Author
import Author.Username as Username
import Effect exposing (Effect)
import Expect
import Session
import Test exposing (..)


effectToMethodAndUrl : Effect msg -> Maybe ( String, String )
effectToMethodAndUrl effect =
    case effect of
        Effect.FetchAuthor _ toRequest ->
            let
                request =
                    toRequest Session.guest
            in
            Just ( request.method, Api.Endpoint.unwrap request.url )

        _ ->
            Nothing


api : String
api =
    "https://conduit.productionready.io/api/"


all : Test
all =
    describe "HTTP requests"
        [ test "fetch an author" <|
            \_ ->
                Effect.fetchAuthor identity (Username.fromString "IainMBanks")
                    |> effectToMethodAndUrl
                    |> Expect.equal (Just ( "GET", api ++ "profiles/IainMBanks" ))
        ]

You can notice that we are able to inspect the elements we want to test from the Effect.

What about elm-program-test?

I have never used yet avh4/elm-program-test, but it seems to be similar, as it requires to define a simulated perform function that returns simulated effects to be able to completely simulate the application, including its update function:
https://elm-program-test.netlify.app/cmds.html#simulating-effects

One issue is that elm-program-test is based on the principle that effects only represent Cmd, not state changes, so the effects inducing a model update would not be testable with it.

Another issue to use elm-program-test is the Browser.Navigation.key, which apparently requires some non trivial modifications to the application before being able to test it:
https://package.elm-lang.org/packages/avh4/elm-program-test/latest/ProgramTest#createApplication

Note that the current elm-realworld-example-app source code has many opaque types and required to expose more things to be able to write the above test. This is a common issue with Elm and tests.

4

This is very neat! I wonder if some of the routing complexity (I mean specifically the existence of onUrlRequest option) can be subsumed into this.

Also I wonder if there would be architectural benefits in giving subscriptions a similar treatment?

1 Like
5

@dmy Ah got it. Thanks for the explanation.

1 Like
6

This is a thing of beauty. I’m focused on the state for now - you say

Effect pattern , is most often described only in the context of tests and Cmd but it can be extended to state changes

At a conceptual level is this because you are dealing with state outside the scope of the page / module or whatever and changes in that state are therefore in the world of effects?

At a practical level, in an environment in which I cannot use a custom application (because one is already in use) can the pattern be applied?

2 Likes
7

This is how I see it indeed.

A page update function does not have to know if it requests a Cmd or a state change at the top level.

From any update function perspective, including Main.update, anything that has side effects is an Effect (well, except turning Html msg into a DOM, but it’s not requested by the update function).

The fact that the effect is actually handled by the Elm runtime or by the unexposed Effect.perform function that acts as the application runtime does not matter.

This is not a problem, you can notice in elm-realworld-example-app that Effect.application actually calls another one, Api.application, that handles the decoding of credentials from the flags. So you can nest applications, each one providing some different features.

1 Like
8

I’m not sure to understand. This makes onClick routing easier from subpages, but what is complex about onUrlRequest and why do you think it could help?

Subscriptions trigger a message in the update function that can return an Effect, for example for a local storage session change in this application, so they already use effects indirectly. Do you think about something wrapping subscriptions?

9

One issue I had with elm-spa architecture was going from one Page to another with information that couldn’t be encoded in the URL. I think part of that problem was the directionality of that information, where one goes from URL to message to state. I was wondering if this pattern could reverse the direction and go from message to state to url.

What I mean by reducing complexity is that personally I always find the onUrlRequest and onUrlChange options in Browser.application pretty confusing. Thankfully it’s the sort of thing one usually sets up at the beginning and then doesn’t mess with much, but I feel that the routing APIs could use a bit of streamlining.

1 Like
10

I see, thanks for clarifying. I think that this is almost exactly what Effect.logout does. It first updates the session in the model, then requests the URL change .

So we get Msg → Effect → Model update → replaceUrl → onUrlChange → Page changed

A drawback compared to onUrlRequest is that it requires at least one Msg per page to handle clicks.

11

Thanks very much for the explanations - I’ve got to get my head around the second one (in practical terms, but it has certainly opened my eyes. And thanks for your hard work on this - it is super-helpful to have the documentation presented next to the example in that way - a fantastic learning resource.

1 Like
12

Thank you! It was a lot more work than I expected at first to get things clean enough to share them :sweat_smile:

If this can help about Effect.application, I could add that it is not strictly required anyway. You could remove it, expose the Effect.perform function and call it yourself on your init and update functions in your Browser or custom application call from your main. Something like:

Custom.application
    { init = \flags url key -> init flags url key |> Effect.perform Ignored
    , view = view    
    , update = \msg model -> update msg model |> Effect.perform Ignored
    , subscriptions = subscriptions    
    , onUrlChange = onUrlChange    
    , onUrlRequest = onUrlRequest    
    }

I think it’s cleaner and safer though to nest them and encapsulate everything in the Effect module .

13

I really like how this approach frees you from pasing things like session tokens around down to places that don’t even care about them a lot. Very nice :smiley:

1 Like
14

Some effects still require the Cred that include the authentication token, for example Effect.favorite, but this is just to prevent the developer from calling them unauthenticated. The actual token is not really required as this point.

For those that work both authenticated and unauthenticated, like Effect.fetchAuthor (that will fetch the “followed” boolean if authenticated), the token is not required and added by the effect handler indeed.

A third case is Effect.fetchSettings, that does not require a Cred for practical reasons (it is used in an init function), but only works if authenticated, so in this case the effect handler will redirect to the login page if the session is unauthenticated.

So this gives quite a lot of freedom indeed.

15

I am looking for an example where this pattern replaces the outMsg, or SharedState style - updating state that is shared between pages - in the example even session seems to be based on the Api rather than a model update - The only thing I can see is a replacing of the session in the top model rather than an update to the session - would you be able to point out something that am I missing in the example app, or elsewhere, please?

16

It’s true that the only thing that needs to be updated from subpages in this application is the session, because each page loads its own non-shared data. Different applications may have different needs.

I don’t understand what you mean by “rather than an update to the session”. The new session is decoded indeed from an HTTP Api response to Effect.authenticate, but the session is stored anyway in the Main.Model and it needs to be updated with the new one from the subpage that did the request, once authenticated. So Effect.login seems to me to be a fine example of updating a state shared between pages.

The Page module (which does not use the same Msg than Main) also updates the Main.Model.session by calling Effect.logout that will replace the top model session by an anonymous one.

Lastly note that because of this intermediate Page, Page.Login is actually a sub-sub-page, so it would require an OutMsg at two levels to update the session.

I have an application with tens of those, because the application pages are basically different views and editors of the same shared data, and they are very similar to these examples. So I’m not sure what you expect by looking for more examples.

2 Likes
17

hi, how do i set subscriptions in subpages? elm-spa-example does it in subscriptions section of Main.elm. i can not figure out how to set it, i do need it for AWS Cognito signin. thanks

18

You do it the same way as in elm-spa-example, except there is an additional Page.subscriptions because of the way I organized pages.

For example to add subscriptions to the login page:

In Main:

subscriptions : Model -> Sub Msg                                                                                                                                                                                                
subscriptions model =                                                                                                                                                                                                           
    Sub.batch                                                                                                                                                                                                                   
        [ Session.onChange GotSession                                                                                                                                                                                           
        , Sub.map GotPageMsg (Page.subscriptions model.page)                                                                                                                                                                    
        ]

In Page:

subscriptions : Page -> Sub Msg                                        
subscriptions page =              
    case page of                  
        Login login ->            
            Sub.map GotLoginMsg (Login.subscriptions login)      
                                  
        -- Wildcard to shorten the example but you can use an exhaustive pattern matching
        _ ->  
            Sub.none

and finally in Page.Login:

subscriptions : Model -> Sub Msg    
subscriptions model =
    Sub.none

I have added a with-login-subscriptions branch with it if you want to check it.
Here is the commit:

19

Excellent. Thank you very much. I could have figured it out by myself… Thanks again.

1 Like
20

Thanks for sharing!

Would you say that the main difference between the OutMsg pattern and the Effect pattern is that in OutMsg, we only define effects for sub-pages, but with Effect, we define them for the main update function as well?

I feel that there is an important idea here that I haven’t been able to formalize, but essentially comes down to the idea that the main way to compose Elm applications is by recapitulating the Elm architecture at each level (each page gets its own model, view, and update). In this way, each “page” or “subapplication” or even “component” is being managed by the larger application in the same way that the application is itself being managed by the Elm runtime. This seems pretty logical to me although it seems like common practice instead is to almost recapitulate the Elm architecture, but with e.g. a giant global shared Effect type.

I think it is really interesting to think about what kinds of things “belong” at each level, what needs to get passed down and what needs to get passed back up. As an example, sometimes when a component or subapplication wants to request some action from its container, the container needs to provide a way to “respond” and inform the subapplication about things that happened. The toplevel Elm runtime leans heavily on a toMsg pattern because it has to be generic and not know anything about your application – the only way it can provide information to you is by calling update. This also has the nice property that in each subapplication, the Msg type can be opaque.

However, the coupling is much closer between your main application and a subapplication, and I think this admits other ways of structuring things. For example, the fetchAuthor effect is only used in one place by one subapplication – or in other words, there’s only one concrete value for its first argument – so instead of passing that toMsg around, the containing application could just use the subapplication’s CompletedAuthorLoad message directly.[1] EDIT to add: Obviously there are cases where tho toMsg pattern makes sense because you want to express the same effect but have the interpretation of its response be different sometimes.

And of course there are cases where you don’t even need a Msg – the subapplication can just offer functions Model -> Model that the containing application can call directly. In my application, I do this to inject information when a request is made into the subapplication so that it knows what to wait for.

These kinds of ideas are all very abstract and I find it hard to describe them verbally so I think writing them up in code is a great idea, so again thanks for sharing!

[1] I guess this isn’t actually possible given the architecture you have now, where Effect is depended on by all subapplications, so if Effect refers to a subapplication, you have an import cycle. In my application I have the dependencies go the other way, where each subapplication defines its own Effect (which I call Cmd), and the containing application can refer “inside”.