Use cases for effect manager

Hi. I’d like to request a feedback from community to explore “alternative” (or extended) architecture on top of TEA. In a company where I work I had an opportunity to develop frontend architecture for our web app. We took a path of migrating existing AngularJS web app to Elm in a smart way (means reuse existing AngularJS code when possible and rewrite it when time comes). It allows us to focus on new features and benefit from new technologies such as Elm.

Quickly I realized that it’s possible to extend TEA by abstracting common functionality into an extended runtime. Basically it’s possible to add a layer between TEA and business logic. As an example I showed here how it’s possible to add missing functionality of un mounting an Elm program. So instead of complaining that Elm sucks one can just extend the language to ones needs. Yes Elm needs your love, because it’s still young kid which is growing (parents will understand me)!

So it appears that Elm is strict and “limited” but it can be extended to infinity and beyond… =) One possible way to extend it is by creating a layer between TEA and end logic. At our company an extended runtime is quite big (3k LoC, though it’s in a process of being split into smaller modules). It abstracts a lot of common functionality, such as animation, popups, form states, common data, etc… (basically what makes your app uniq).


Now to the point of the topic. How do we communicate with runtime? The most common approach is a so called Taco approach. Usually modules expose an update function with ExternalMsg or something similar. We called it Action’s. But with time I realized that those are actually side Effect’s (or in Elm we call them managed effects, because Elm does not support side effects).

From Wiki:

In computer science, an operation, function or expression is said to have a side effect if it modifies some state variable value(s) outside its local environment, that is to say has an observable effect besides returning a value (the main effect) to the invoker of the operation.

In Elm effects are managed with Cmd’s (or Task’s). Platform module from core packages mentions following.

Effect Manager Helpers

An extremely tiny portion of library authors should ever write effect managers. Fundamentally, Elm needs maybe 10 of them total. I get that people are smart, curious, etc. but that is not a substitute for a legitimate reason to make an effect manager. Do you have an organic need this fills? Or are you just curious? Public discussions of your explorations should be framed accordingly.


I’d like to challenge the statement that Elm needs maybe 10 of them total. Is that really true or should developers be allowed to have domain specific effects. I will give examples based on app we develop at Conta.


When do we need effects?

There are probably various situations when effects are needed. I’ll talk about one of them. We may need an effect when we would like to modify external state. Let’s imagine we have a web app with many pages (e.g. facebook). In our app we have a popup system. A popup always looks the same, it probably have some overlay to hide content of a page and some frame for popup. The different part is the content of a popup. At least our popup should have a state Open | Closed. In Elm we have a single source of truth, so a popup can not have it’s state, the state of a popup should be stored somewhere.

A simplest approach would be to store the state directly in a module where we want to show a popup. It has at least two disadvantages for my taste. (1) The Model is polluted with data we are not actually interested in, we are interesting only in business data at most. (2) The state is duplicated (should be repeated) for each page, msg and update function would have additional expressions copied again and again.

Another approach would be to store a popup state at our extended runtime. So whenever we want to show a popup we can do so with an effect (external message in Taco approach). We provide a content of a popup and our extended runtime handles the state of a popup. That’s a side effect which is managed by our extended runtime.

That’s a very simple example how a business can have it’s domain specific effect which can not be handled by Elm runtime out of the box. And there is no way to support all domain specific effects. Such effects are pure Elm effects and does not require native code.

Luckily we are smart developers and we can add missing features when we need them. At the moment we have 3 domain specific effects for our app (animation, popup, form handling). Though there more will come in the future.

My question are

  • Doesn’t it make sense to open support for custom domain specific effects which does not require native code for Elm runtime instead of extending the language.
  • Why are effect managers restricted now? Is that because they require native code?
  • Can we support pure Elm effects?

As an appendix I’ll add our current implementation of effects module.

module Shared.Kernel.Effects exposing (Effects, apply, applyWithPriority, batch, from, map, none)

{-| In Elm all effects are managed (no side effects).
Effects are described with commands and subscriptions (Cmd msg, Sub msg).
Only private organisations/people have access to add/describe new effects.

Effects module helps to manage additional program effects.
For instance effects can be used for communication between programs.
It make sense especially for programs described as a composition of sub-programs.
A program effect should be supplied by a consumer.

-}


{-| Describes a collection of program effects.
-}
type Effects effect
    = Effects (List effect)


{-| Creates no effect. Useful when a program does not produce any effects.
-}
none : Effects effect
none =
    Effects []


{-| Creates an effect. Useful to create an instance of a program effect.
-}
from : effect -> Effects effect
from =
    Effects << List.singleton


{-| Batches multiple effects. Useful when a program produces multiple effects.
-}
batch : List (Effects effect) -> Effects effect
batch =
    Effects << List.concat << List.map toList


{-| Transforms an effect. Useful when combining multiple programs together.
-}
map : (effectA -> effectB) -> Effects effectA -> Effects effectB
map tagger (Effects effects) =
    Effects (List.map tagger effects)


{-| Applies all effects to given initial value.
Useful to apply effects produced by a program.
-}
apply : (effect -> value -> value) -> value -> Effects effect -> value
apply applyEffect initialValue (Effects effects) =
    List.foldl applyEffect initialValue effects


{-| Applies all effects to given initial value by given priority.
Useful to apply effects produced by a program.
Effects with lover priority are applied first.
-}
applyWithPriority : (effect -> value -> value) -> (effect -> Int) -> value -> Effects effect -> value
applyWithPriority applyEffect priority initialValue (Effects effects) =
    effects
        |> List.sortBy priority
        |> List.foldl applyEffect initialValue


{-| Converts effects into a list.
-}
toList : Effects effect -> List effect
toList (Effects effects) =
    effects

Basically it abstracts the functionality to batch effects, map them and fold. That’s the minimum we need. An extended runtime then implements the logic to update the state whenever a child module returns an effect from it’s update function. Please let us know if you’d like us to publish it as a package.

Additionally I’ll provide an animation module which uses effects module.

module Shared.Kernel.Animation exposing
    ( Effects
    , Model
    , Msg
    , Name
    , Transition(..)
    , Type(..)
    , animateIn
    , animateOut
    , animatingIn
    , animatingOut
    , apply
    , idling
    , init
    , name
    , render
    , subscription
    , update
    )

{-| Animation system is used to store, update and render all animations.
It uses effects for communication with other systems.
It describes all animation types and styles.
-}

import Accessibility as Html
import Animation
import Animation.Messenger as Messenger
import Dict exposing (Dict)
import Shared.Extra.Animation as Animation
import Shared.Extra.Cmd as Cmd
import Shared.Kernel.Effects as Effects
import Time



-- MODEL


{-| Represents animation model which stores all the animations provided by other systems.
-}
type Model msg
    = Model (Dict String (Animation msg))


{-| Initialises animation model from given effects.
-}
init : Effects msg -> Model msg
init =
    apply (Model Dict.empty)


{-| Applies given effects to existing animation model.
-}
apply : Model msg -> Effects msg -> Model msg
apply =
    Effects.apply applyEffect


{-| Takes an animation name and returns list of html attributes for current animation.
-}
render : Name -> Model msgA -> List (Html.Attribute msgB)
render (Name animationName) (Model model) =
    Dict.get animationName model
        |> Maybe.map (Animation.render << style)
        |> Maybe.withDefault []



-- EFFECT


{-| Animation system effects which are used for communication with other systems.
-}
type alias Effects msg =
    Effects.Effects (Effect msg)


{-| Animation system effect. Describes what other systems can ask for from animation system.
-}
type Effect msg
    = Animate Type Direction Transition Name (Maybe msg)


{-| Applies animation system effect to current animation model.
-}
applyEffect : Effect msg -> Model msg -> Model msg
applyEffect (Animate type_ direction transition (Name animationName) changeMsg) (Model model) =
    Model (Dict.update animationName (animate type_ direction transition (Name animationName) changeMsg) model)


{-| Creates an effect to start animation in.
-}
animateIn : Type -> Transition -> Name -> Maybe msg -> Effects msg
animateIn type_ transition animationName changeMsg =
    Effects.from (Animate type_ In transition animationName changeMsg)


{-| Creates an effect to start animation out.
-}
animateOut : Type -> Transition -> Name -> Maybe msg -> Effects msg
animateOut type_ transition animationName changeMsg =
    Effects.from (Animate type_ Out transition animationName changeMsg)



-- TRANSITION


{-| Represents animation transition.
-}
type Transition
    = Instant
    | Smooth



-- ANIMATION


{-| Represents an animation itself.
-}
type Animation msg
    = Animation Name Type State (Style msg)


{-| Animates an
-}
animate : Type -> Direction -> Transition -> Name -> Maybe msg -> Maybe (Animation msg) -> Maybe (Animation msg)
animate type_ direction transition animationName changeMsg =
    Just
        << Animation animationName type_ (Animating direction)
        << animateStyle type_ direction transition animationName changeMsg
        << Maybe.withDefault (Animation.style [])
        << Maybe.map style


{-| Idles an animation.
-}
idle : Animation msg -> Animation msg
idle (Animation animationName type_ _ animationStyle) =
    Animation animationName type_ Idle animationStyle


{-| Checks if animation is idling.
-}
idling : Name -> Model msg -> Bool
idling (Name animationName) (Model model) =
    Dict.get animationName model
        |> Maybe.map (state >> (==) Idle)
        |> Maybe.withDefault True


{-| Checks if animation is animating in.
-}
animatingIn : Name -> Model msg -> Bool
animatingIn (Name animationName) (Model model) =
    Dict.get animationName model
        |> Maybe.map (state >> (==) (Animating In))
        |> Maybe.withDefault False


{-| Checks if animation is animating out.
-}
animatingOut : Name -> Model msg -> Bool
animatingOut (Name animationName) (Model model) =
    Dict.get animationName model
        |> Maybe.map (state >> (==) (Animating Out))
        |> Maybe.withDefault False



-- NAME


{-| Represents an animation name. It's just a wrapper around string with to achieve type safety.
-}
type Name
    = Name String


{-| Creates an animation name.
-}
name : String -> Name
name =
    Name



-- TYPE


{-| Represents an animation type. A behaviour of animation.
-}
type Type
    = Fade
    | SlideDown



-- STATE


{-| Represents an animation state.
-}
type State
    = Idle
    | Animating Direction


{-| Returns an animation state.
-}
state : Animation msg -> State
state (Animation _ _ animationState _) =
    animationState


{-| Represents an animation direction.
-}
type Direction
    = In
    | Out



-- STYLE


{-| Represents an animation style which is internally a collection of animation properties.
-}
type alias Style msg =
    Messenger.State (Msg msg)


{-| Returns styles for all animations.
-}
styles : Model msg -> List (Style msg)
styles (Model model) =
    List.map style (Dict.values model)


{-| Returns an animation style.
-}
style : Animation msg -> Style msg
style (Animation _ _ _ animationStyle) =
    animationStyle


{-| Animates an animation style. Each transition and direction produces a list of animation steps.
-}
animateStyle : Type -> Direction -> Transition -> Name -> Maybe msg -> Style msg -> Style msg
animateStyle type_ direction transition animationName changeMsg =
    case transition of
        Smooth ->
            case direction of
                In ->
                    Animation.interrupt
                        [ Animation.set (property type_ Out)
                        , Messenger.send (changed changeMsg)
                        , Animation.wait (Time.millisToPosix 50)
                        , Animation.toWith (interpolation type_) (property type_ In)
                        , Messenger.send (Animated animationName)
                        ]

                Out ->
                    Animation.interrupt
                        [ Animation.set (property type_ In)
                        , Animation.toWith (interpolation type_) (property type_ Out)
                        , Animation.wait (Time.millisToPosix 50)
                        , Messenger.send (changed changeMsg)
                        , Messenger.send (Animated animationName)
                        ]

        Instant ->
            Animation.interrupt
                [ Animation.set (property type_ direction)
                , Messenger.send (changed changeMsg)
                , Messenger.send (Animated animationName)
                ]


{-| Represents an animation property. It's beginning and end of an animation.
-}
property : Type -> Direction -> List Animation.Property
property type_ direction =
    case ( type_, direction ) of
        ( Fade, In ) ->
            [ Animation.opacity 1 ]

        ( Fade, Out ) ->
            [ Animation.opacity 0 ]

        ( SlideDown, In ) ->
            [ Animation.translate (Animation.px 0) (Animation.px -5) ]

        ( SlideDown, Out ) ->
            [ Animation.translate (Animation.px 0) (Animation.px -1000) ]


{-| Represents an animation interpolation based on animation type.
-}
interpolation : Type -> Animation.Interpolation
interpolation type_ =
    case type_ of
        Fade ->
            Animation.ease300

        SlideDown ->
            Animation.ease600



-- UPDATE


{-| Represents animation messages.
-}
type Msg msg
    = NoOp
    | AnimationMsg Animation.Msg
    | Changed msg
    | Animated Name


{-| Represents changed animation message.
-}
changed : Maybe msg -> Msg msg
changed =
    Maybe.map Changed >> Maybe.withDefault NoOp


{-| Updates an animation model based on given animation message.
As a result it produces a tuple of three elements, new animation model, animation command, and external command.
-}
update : Msg msg -> Model msg -> ( Model msg, Cmd (Msg msg), Cmd msg )
update msg (Model model) =
    case msg of
        NoOp ->
            ( Model model, Cmd.none, Cmd.none )

        AnimationMsg animationMsg ->
            -- Ignoring dict key because it's not used
            Dict.foldl (\_ -> updateAnimation animationMsg) ( Model model, Cmd.none, Cmd.none ) model

        Changed changeMsg ->
            ( Model model
            , Cmd.none
            , Cmd.dispatch changeMsg
            )

        Animated (Name animationName) ->
            ( Model (Dict.update animationName (Maybe.map idle) model)
            , Cmd.none
            , Cmd.none
            )


{-| Updates an animation.
-}
updateAnimation : Animation.Msg -> Animation msg -> ( Model msg, Cmd (Msg msg), Cmd msg ) -> ( Model msg, Cmd (Msg msg), Cmd msg )
updateAnimation msg (Animation (Name animationName) type_ animationState animationStyle) ( Model model, cmd, externalCmd ) =
    let
        ( newAnimationStyle, animationCmd ) =
            Messenger.update msg animationStyle
    in
    ( Model (Dict.insert animationName (Animation (Name animationName) type_ animationState newAnimationStyle) model)
    , Cmd.batch [ cmd, animationCmd ]
    , externalCmd
    )



-- SUBSCRIPTION


{-| Represents animation system subscription.
-}
subscription : Model msg -> Sub (Msg msg)
subscription model =
    Animation.subscription AnimationMsg (styles model)

An animation module provides animation effects, model, subscription, update. So it’s very possible to extend Elm and add support for effect managers. If Elm runtime would allow us to create domain specific effects wiring would be slightly easier, because update function would be called automatically. Am I correct?

Thank you,
Andrey.

2 Likes

The websockets package used effect managers without any native code so… it is not because they require native code. Effect managers allow for state outside of the model and I guess this is what was considered way to dangerous for regular use.

Yes, but this is not officially supported. Effect mangers are considered an implementation detail that might change and as such, it is not open for the general public in order to not create difficulties for the core team when they want to change the implementation.

The closest thing to this is an internal package that is not exposed in elm.json. It is designed to be used by the package author but the outside world is not suppose to know or care about its existence. It is an implementation detail. With 0.18 it was like the internal package was accidentally exposed and when the move to 0.19 happened, it was removed from the exposed modules. It still works the same but the new way expresses in a clear way that this is a private, implementation detail that should not be used outside the package.

The ExternalMsg is not a side-effect, it is a return value from the pure update function. I used to think that way too, but now I see there is only one side-effecting function in an Elm program, and that is the top-level update function. Any other update function, is just a function - like when you split a big function down into smaller ones because it is getting too large.

I think it would be totally fine for Elm to allow developers to write their own effects modules including kernel code - just so long as they cannot be published. Banning this was too heavy handed given the restrictions on publishing that were already in place; we’re all grown ups it ok for people to go their own way, just so long as they really are going their own way. Early in my time with Elm, I wrote an effects manager, but when I realised I could not publish it and that more of my code was dependent on it and so could also not be published, I also realised that by pursuing that I was shutting myself off from the rest of the community.

Everything that is published relies on a small and very well thought out kernel. Everything that is published is also stateless. Those are the strong gaurantees that make Elm so attractive to work with - All we can share is pure functions, no risk of getting some hidden state along with it.

So you have a way of holding state in a Dict in order to keep the model ‘clean’ and focussed on the business logic, fair enough if that is how you like to organize things. I am just wondering though, given that this is animation state and therefore updated at 60 frames/sec, is the Dict.update going to become a bottleneck? I guess not, unless you are really doing massive amounts of animation…

Here is a simple Ellie with a simple two page example. It shows how effects module can be used to extend Elm program with additional managed effects. The routing is done in main function via button click to simplify things, normally that would be handled by router. An example also contains an anti-pattern by storing Html msg in a model. That can be avoid but for simplicity is done in that way. Another note is that runtime is abstracted away from Main in order to keep main module clean and pure plumbing. All of the runtime can be moved to main module. Also when runtime is separate it allows to render both full app and single page easily.

Once again such approach works for quite complex apps well, and probably an overkill for small apps. Another great thing is that it can work very well with a program test package. Because all Cmd's can be abstracted as effects. So it’s a nice field to explore I think.

Yes, an ExternalMsg itself is not a side effect. But the update function call of that msg is a side effect in terms of caller. E.g. I’m as a caller does not care how animation will be done, I just want to start in and get the final result.

In our cases performance is good enough, it’s just simple fade/slide animations. Have not done any measurements.

I’d like to clarify my statements since I’ve got feedback from the community. My initial thought was to understand basically if i’m going the right direction. I’ve found out that additional domain specific effects make sense. Though current implementation of effect manager is not safe to be exposed for the folks to use. Because one can create runtime exceptions with invalid implementation. Elm as a language does not include exception handling by eliminating exceptions at all. I’ve got advice to learn about stateful processes and how they can help to have domain specific effects. Maybe I can come up with some interesting ideas in the future.

Anyway I wanted to share what I found and it’s a neat way to architect complex web apps imho. My intention is definitely not to blame current implementation but to explore new possibilities of future growth for the Elm language.

1 Like

Possibly related? https://package.elm-lang.org/packages/webbhuset/elm-actor-model/latest/

@stoft Why do you think so?

The way I see it, you have re-invented effects managers but Elm forces you to make your state explicit, which is fine - call it ‘effects-managers-with-explicit-state’ if you like. If its working well for you, keeping your code clean, and helping with code re-use by allowing common functional blocks to be built and applied, then its obviously working well and doing good stuff.

I also think the 0.18 elm-mdl package worked along these lines, effect state was held in its Mdl model, which had to be explicitly wired into your own application Model, update and view. I have a related module, lifted mostly from elm-mdl, for managing UI effects, bit of a work in progress so not exposed in the package, but here it is: https://github.com/the-sett/the-sett-laf/blob/6.1.0/src/TheSett/Component.elm

I’ve updated the example and added a transition function between pages. So now the shared data actually shared between pages. On the first page it’s possible to modify shared data with effect and the second page will reuse it.

Yes @rupert. Keeping state explicit is actually a good thing, that avoids side effects. Maybe saying it’s an extended TEA is not right, because it’s still the same TEA. But the fact that it’s possible to manage effects explicitly is neat.

1 Like

If you treat commands from update functions as simply a particular flavor of “thing you want other code to do for you and possibly get back to you”, then you can use alternative types in this role that can be interpreted by Elm code further up the model tree. This provides a way to write pseudo-effects managers in Elm.

Such an approach could be used as a path to moving native code into Elm. Put all of the interaction behind a pseudo-command-type based API. At first, interpret this API at the top level of your program by messaging to the native code using ports. Then write an Elm version to handle the API without needing to touch the API clients.

Doing this, however, will likely involve storing functions in the model, so if you feel that’s a horrible thing to do, then you won’t want to take this path.

It’s also difficult to hook into the subscriptions me mechanism since at the point when subscriptions is called, you can’t make changes in the model and hence can’t make changes in your pseudo-effects-manager model.

1 Like

You somewhat overstate the safety that comes with sharing only pure functions. True, their side-effects are constrained and they probably can’t readily damage your other data. A malicious package could, however, make Http calls and you wouldn’t know it from the source code because commands are a black box.

(If one didn’t want this hole, update functions would return requests for action would be interpreted into external effects by application code. So, for example, a module could return a value indicating that it wanted a particular Http request performed, but that would get interpreted into a command at the top level of the application.)

I haven’t had time to dig into the details of this very long thread so you’ll have to excuse me if I’m off target, but you are talking about message passing and encapsulating state, which to me sounds very similar to the actor model. :slight_smile:

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.