If Elm had threads..?

[lpil]

@rupert

Where is the actor model here though ? This looks like coroutines, but I do not see an actor and message passing ? But perhaps that is implicitly there, I just do not know how to read it.

Actors are just coroutines with message passing and typically without function colouring. Like all concurrency patterns, the programmer is free to build abstractions on top of them.

@rupert

Looking in Elixir docs I see this example, which looks more actor like to me, receiving a message and processing it.

defmodule Example do
  def explode, do: exit(:kaboom)

  def run do
    Process.flag(:trap_exit, true)
    spawn_link(Example, :explode, [])

    receive do
      {:EXIT, _from_pid, reason} -> IO.puts("Exit reason: #{reason}")
    end
  end
end

This will be an example that explains the most low level primitives of the concurrency system, specifically it will be teaching how exits propagate via links and how an actor can trap exits.

In a production Elixir application you will almost never see code like this, the higher level abstractions will always be used. e.g. GenServer, Supervisor, Task, etc.

To use the low level primitives directly means choosing to reimplement what the abstractions give you for free. The equivalent in Elm would be using Platform.worker and then writing your own virtual DOM in order to modify the page via ports.

@rupert

Here is a revised messaging API. This one combines P2P and a separate PubSub that I had, into a single API for Channels.

Why have publish at all? Isn’t that just List.map (send message) channels?

Your API doesn’t have link or monitor, does that mean the expectation is that a task is never poisoned or panics? If so, how will you deal with state corruption, diverging tasks, and deadlocks?

[rupert]

@lpil

Actors are just coroutines with message passing

Not strictly true. Actors and coroutines are concepts that exist on slightly different levels.

Actor model is a concurrency architecture and computation model.
Coroutines is a control-flow mechanism and language feature.

So actors could be implemented using coroutines, and that is probably what you are used to on the BEAM. Actors could also be implemented using threads, or on separate nodes on the network, or…

results = 1..n
  |> Enum.map(&Task.async(fn -> do_work(&1) end)
  |> Task.await_many()

We could do couroutines in Elm, and the code would look not entirely unlike this. But as I pointed out above, Elm Task cannot run an Elm Cmd, which restricts what side-effects can be run, compared with the TEA-like model.

[rupert]

@lpil

Your API doesn’t have link or monitor, does that mean the expectation is that a task is never poisoned or panics? If so, how will you deal with state corruption, diverging tasks, and deadlocks?

Right. Well, that is the purpose of this conversation - to explore what might be needed to make the API usable.

Process monitoring would logicall seem to belong in the Actore.Core API I posted, since it is at the level of the Process.

[lpil]

@rupert

Not strictly true. Actors and coroutines are concepts that exist on slightly different levels.

Actor model is a concurrency architecture and computation model.
Coroutines is a control-flow mechanism and language feature.

So actors could be implemented using coroutines, and that is probably what you are used to on the BEAM. Actors could also be implemented using threads, or on separate nodes on the network, or…

To avoid getting into an argument of semantics: I will be talking about the mainstream style of actors where they are coroutines. It is possible to implement actors as threads or as distinct programs, but they would share little-to-no semantics with coroutine based actors, and I’m not familiar such a system.

I suppose a HTTP microservice written in a non-async language such as Ruby could be considered an OS process based actor. That’s fun to think about

@rupert

Right. Well, that is the purpose of this conversation - to explore what might be needed to make the API usable.

Process monitoring would logicall seem to belong in the Actore.Core API I posted, since it is at the level of the Process.

There’s an interesting tension in Gleam as it largely is a no-crash language (though not as much as Elm!), yet crashing is a key part of how OTP deals with these and how it achieves its fault tolerant properties. I’d be cool to explore other solutions here.

@rupert

type Channel c a = Channel

You’ve gone for the name channel. Does that mean they do not “belong” to a specific actor and they can be consumed by multiple concurrently?

[rupert]

@lpil

Why have publish at all? Isn’t that just List.map (send message) channels?

@lpil

@rupert

type Channel c a = Channel

You’ve gone for the name channel. Does that mean they do not “belong” to a specific actor and they can be consumed by multiple concurrently?

Yes. Although that was already true for Subject, I renamed to Channel to be neutral on whether the mailbox is a p2p or a pubsub.

To answer the first question, why have publish at all? Just List.map (send message) channels.

The answer is that publish does not surface a need to do send N in the API. This means that the implementation is free to optimize this. You could send just 1 message into a buffer, but have N readers, each with their own index into the buffer. So they can independantly read the same message. So the fan-out happens on the reader side, not the sender side. So I am trying to abstract away implementation details to maximize opportunities for optimisations behind the scenes.

[rupert]

@lpil

There’s an interesting tension in Gleam as it largely is a no-crash language (though not as much as Elm!), yet crashing is a key part of how OTP deals with these and how it achieves its fault tolerant properties. I’d be cool to explore other solutions here.

Yes, Elm is strong on that. But there is always the possibility of overflowing the stack and I don’t see that ever going away. There are other bugs in Elm that can cause crashes, like modBy 0, but those could be fixed. But still, so long as there is at least one way of crashing, fault recovery is needed. And as you say, there could be deadlocks or livelocks too.

What is the Erlang or Elixir API around this ? What could it look like in Elm ?

[wolfadex]

Have you considered looking at Pony as a reference for actor model that’s different from BEAM ones?

[rupert]

No.

Example here: Pony

Looks like message passing is written just like a “method call” on a function, in the sense that one actor is invoking a be (whatever that is) on another directly, instead of doing a send.

Not sure how this would translate into Elm though ?

[rupert]

Something like this for process supervision ? OTP inspired.

module Actor.Core exposing (..)

type Actor flags model msg
    = Actor


type Process msg
    = Process


type ExitReason
    = Normal
    | Shutdown
    | Crashed


actor :
    { init : flags -> ( model, Cmd msg )
    , update : msg -> model -> ( model, Cmd msg )
    , subscriptions : model -> Sub msg
    }
    -> Actor flags model msg
spawn : Actor flags model msg -> flags -> Task x (Process msg)
self : Task x (Process msg)
exit : Process msg -> ExitReason -> Task x ()
kill : Process msg -> Task x ()

and

module Actor.Supervisor exposing (..)

type Supervisor msg
    = Supervisor


type ManagedActor msg
    = ManagedActor


type Strategy
    = OneForOne
    | OneForAll
    | RestForOne


type Restart
    = Permanent
    | Transient
    | Temporary


type Event msg
    = ActorStarted (Process msg)
    | ActorExited (Process msg) ExitReason
    | ActorRestarted (Process msg)


managedActor : Restart -> Actor flags model msg -> flags -> ManagedActor msg
supervisor : Strategy -> List (ManagedActor msg) -> Supervisor msg
start : Supervisor msg -> Task x (Process msg)
events : Supervisor msg -> Sub (Event msg)

[wolfadex]

My thinking is more that Pony might be closer to what you’re looking for in terms of “actor based language that’s fully typed and doesn’t crash”. It’s been some years since I looked at it so I don’t know how it compares to Gleam, but it could be another reference point for how to design things in terms of actors.

[rupert]

Here is an implementation of these APIs on top of elm-procedure. Clean API models are in the model/ folder. The implementations are inevitably less clean since they need a bit of plumbing around elm-procedure. The model APIs would need kernel functions or effects modules to implement them as written.

There are real working examples that you can run too:

GitHub - rupertlssmith/elm-with-threads: What would Elm with threads be like?

The Kafka inspired Topic stuff - I might have gone a bit overboard with this… but one of the ideas there is to have message keys for sharding. So if you have N threads processing messages, you can hash a message key modulo N, and then the flow per key will always go to the same thread, allowing ordering guarantees to be maintained over parallel processed event streams.

The idea is that the system could even scale in or out your Actor without you even having to think about it. Not that I have an API for how to enable that yet, just wanted to explore what the messaging API side of that might look like.

[rupert]

@wolfadex

My thinking is more that Pony might be closer to what you’re looking for in terms of “actor based language that’s fully typed and doesn’t crash”. It’s been some years since I looked at it so I don’t know how it compares to Gleam, but it could be another reference point for how to design things in terms of actors.

I tried making a DSL in Elm out of Pony semantics. By the time I stripped it back to just what can be represented in Elm directly with the actor lifecycle and messaging as APIs, it pretty much just became a TEA like actor implementation with message sending.

In particular Ponys be: clause that defines its behaviours with names that are invoked with messages - that had to be a Msg in Elm, processed by a receive function which was suspiciously like an update.

The thing about Pony is, it builds actor model into the language itself. Without modelling that entire language at the AST level in Elm, and trying to more directly embed it as a DSL, there is not much left that is pony any more.

The capabilities are interesting:

type Capability
    = Iso
    | Trn
    | Ref
    | Val
    | Box
    | Tag

Approximate interpretation of these:

val → plain Elm values
ref → actor-local state (your state)
tag → Address msg (mailbox?)
iso → something like a “linear value passed via spawn/send”
box → read-only API (not enforceable in Elm)
trn → transitional state (hard to model without linear types)

It surfaces some stuff in the language to do with how values are referenced. As far as I can gather the point of this is to make data-race freedom and memory safety provable at compile time without locks.

The problem Pony is solving is that in a normal concurrent system you have shared mutable memory, multiple threads and need for locks, atomics, or discipline. And the hard problem is “How do I know two threads aren’t mutating the same object at the same time?”

Elm is immutable, and that is how it solves that particular problem.