Imitating synchronicity with ports

#1

I have a project with Elm on both client and server. The server is set up with ports to get a HTTP request from Node’s HTTP module with one port, and return a String response using another port:

The Elm part:

port module Server.Main exposing (main)

port httpRequests : (String -> msg) -> Sub msg
port httpResponse : String -> Cmd msg

The JS part:

http.createServer(function (req, res) {

    new Promise(function(resolve, reject) {
        const handler = response => {
          resolve({response, handler});
        };
        app.ports.httpResponse.subscribe(handler);
    })
    .then(obj => {
        res.writeHead(200, {
            'Content-Type': 'application/json',
            'Access-Control-Allow-Origin': '*',
        });
        res.write(obj.response);
        res.end();
        app.ports.httpResponse.unsubscribe(obj.handler);
    })

    app.ports.httpRequests.send(`http://localhost:${port}${req.url}`);

}).listen(port);

This works fine when I’m testing it (there is a lot of time between the requests), but it is, I think, not ideal, as when multiple requests happen “at the same time”, the responses can get switched. Ie. there is probably a race condition.

What to do here? Thread an ID through the ports with the request and response, and implement some kind of “unmatched responses queue” on the JS side, for when the errorneous switch happens? Any ideas?

#2

That model should work well if you create one elm app per request. Thinking of the elm app as an actor that handles one request at a time. So maybe try it out to see if it is performant enough for your use case and you don’t have memory leaks for creating many apps.

If you instead want to think of the elm app as a system that can handle multiple requests at the same time (reusing the elm app across requests like it seems you were doing) I’d encourage you to wire the ports at the top level and only unsubscribe when you close the http server.

Then on the http handler, I can think of a couple of options

  • Either tag request response pair with something and store it, and pass that something to elm to know what messages are linked together
    • Like you mentioned, it could be an ID (auto incrementing int?) or it could even be the request object from the http server as it should be unique per request.
    • You’d have to keep an in memory list of request/responses that are active and waiting with their id and manage that.
  • Another option could be passing the request and response to the elm app (as JSON values), and have the elm app pass it back out, and then JS would just dumbly use that response object to respond.

I actually like that last option, I’m going to try make a small example :smiley:

#3

I managed to write a small example here: https://github.com/joakin/node-elm-server

Unless I’m missing something this should be safe across multiple independent request and free of races. See:

const http = require("http");
const {
  Elm: { Server }
} = require("./elm.js");

const app = Server.init();

app.ports.response.subscribe(([{ req, res }, status, response]) => {
  res.statusCode = status;
  res.end(response);
});

http
  .createServer((req, res) => {
    app.ports.onRequest.send({ req, res });
  })
  .listen(3000);

port module Server exposing (main)

import Json.Decode as D exposing (Value)
import Platform exposing (worker)


type alias Model =
    Int


type Msg
    = Request RequestResponse


type alias RequestResponse =
    { req : Value, res : Value }


type alias RequestInfo =
    { url : String }


main =
    worker
        { init = \() -> ( 0, Cmd.none )
        , subscriptions = \_ -> onRequest Request
        , update = update
        }


port onRequest : (RequestResponse -> msg) -> Sub msg


port response : ( RequestResponse, Int, String ) -> Cmd msg


update : Msg -> Model -> ( Model, Cmd Msg )
update msg count =
    case msg of
        Request ({ req, res } as requestResponse) ->
            case decodeRequest req of
                Ok { url } ->
                    let
                        ( newCount, status, responseText ) =
                            handleRequest url count
                    in
                    ( newCount
                    , response ( requestResponse, status, responseText )
                    )

                Err err ->
                    ( count
                    , response
                        ( requestResponse
                        , 500
                        , "There was an error processing the request"
                        )
                    )


handleRequest url count =
    let
        ok () =
            ( count + 1, 200, String.fromInt count )
    in
    case url of
        "" ->
            ok ()

        "/" ->
            ok ()

        _ ->
            ( count, 404, "Not found" )


decodeRequest : Value -> Result D.Error RequestInfo
decodeRequest value =
    D.decodeValue
        (D.map RequestInfo (D.field "url" D.string))
        value
#4

Yeah, I need the server to be able to hold stuff in-memory and serve multiple clients that can interact with each other through it. Think game server.

And hey, that last option is very nice! It wouldn’t occur to me to send the res object to Elm too. But it works well! :man_shrugging: :smile:

Thanks for the ideas!

#5

I hope that this is a toy project because Elm is not yet suited for server-side work.

For inspiration, I would advise you to take a look at prior work. elm-http-server has an implementation and links to prior projects.

#6

Yeah, it is a toy project.

What’s bad about using it on the server side if I use ports (and not native modules, not available in 0.19 anyway) and embrace the lack of Elm package ecosystem for server side?

EDIT: looked at eeue56/elm-http-server but I don’t believe it’s usable in any form on 0.19 (effect module, native modules)

#7

It’s not that it is “bad” but rather that the concerns of servers are rather more complex.

For a toy example where you make a request and you eventually receive a reply, it is perfectly fine. The troubles would come when you would try to use it in a real world scenario where you have to worry about response times, number of connections, access to the file system, access to databases, various aspects of HTTP protocol, etc. .

#8

Why do you say that? I think Elm is as suitable for server-side work as any Node.js project. It’s just a fancy (and very nice) JavaScript compiler. Perhaps the Node.js version of the Elm runtime hasn’t been exercised enough to work out all the bugs, but that just begs for more users, to find and fix them.

#9

In order to be able to use Elm with the server-side concerns, Elm has to support a series of things otherwise you will get impedance mismatch and things would become rather frustrating.

Without Elm supporting this things out of the box, one is left with

  1. creating a highly complex thing where you would have a lot of code on both sides of the ports
  2. implement support for the missing bits the proper way.

Option 2 requires the willingness to do a domain analysis and design that very few entities are willing (or have the resources) to do. Without the careful analysis and design, any attempt to extend Elm’s support will probably be rejected because it would be against the Elm’s ethos.

There was at least one company that tried to extend the capabilities of Elm within the server-side domain but did not do the analysis and design to a satisfactory degree and ended up with tens of thousands of lines of server-side Elm code that (last that I saw) they decided to abandon.

Code is the easy part, you can implement support for reading files, connecting to databases, handle http requests, etc. quite quickly. Doing it in a way that is long lasting, a way that is easy to use, a way that doesn’t require the API to change frequently… that is not as easy.

#10

I’m not sure how to counter your statement or explain myself properly but I don’t think that is what Elm is from the time I’ve been following it.

That definition is something that fits Bucklescript/Reasonml, typescript, and others, but as far as I understand is not exactly the idea for Elm.

That is what I thought at the beginning too, coming from JS, and it is not a good mindset to have. It will probably lead to frustration and impedance.

Maybe someone else can explain better what I’m saying, I’m having a hard time finding the words.

#11

If you’re thinking of the same people I am, they used a lot of native code, which was all broken by the 0.19 release.

I’ve had success with a using a handful of ports to access Node.js libraries that aren’t part of Elm. But then, none of my code has ever come under big load, and I doubt I’d want to use Node.js at all if it did (I really must spend some quality time with Erlang/Elixir). I view my server side code as a way for mostly-client applications to communicate with each other (multi-player games), or as a way to do something that only the server can do (OAuth authentication). It has worked well for me for that, in the small. I’ll find out how well my jsmaze.com server scales, once I finish upgrading all the libraries it uses, and assuming I get any users.

The big benefit for me with Elm on the server is that I can run the same code on client and server. That was the promise of Node.js when it was first introduced, and I think of Elm as a hugely-better JavaScript to use in that environment.

@joakin I realize that Evan and company designed Elm with the browser environment in mind. I’ve used software well outside its creators’ imagination since I got out of college in the late 1970s. I don’t think I’ll stop any time soon. :wink:

#12

My guess is that they only have a couple hundred lines of code that are trivial to port to 0.19.

Elm is designed to be a nice programming language. The current implementation is focused on the browser but Elm, the language, could just as easily be compiled to run outside of browser. Of course, this would require a different implementation but there is nothing in the language itself to make this difficult.

#13

I disagree. Elm’s LIBRARIES are focused primarily on the browser environment. Elm’s compiler generates standard JavaScript, which is very happy running in Node.js. elm repl relies on that. As does elm-test.

Most people don’t want to run their server code in JavaScript, so they’d prefer output to C or native machine code. But Node.js has a large following, and though I consider it to be a brittle environment for a server, it works, and it pays nearly nothing for context switch or locking, so Node.js servers can handle big loads, as long as they don’t have hairy computation to do.

I wonder how well Elm would work as a language on the BEAM (Erlang’s virtual machine).

#14

We agree on this. The thing is that I view this as an implementation detail not a language design issue.

The folks from Elmchemy are exploring this. :slight_smile:

#15

The way I do it, is to send the callback along with the request (as Json.Value), get an old example here : https://github.com/Warry/simple-server.

While I’m here there is a nice trick to load local files, if you are using the xhr npm module to polyfill the http requests, using the file:// protocol (only works on GET).

#16

I have a port-abstraction library that used to be part of a server framework I was trying to put together for fun. I saved this part of it, and abandoned the rest. I think it would make it really easy to wrap around http.createServer in Node.js

The trick is to pass ids into elm, remember the mapping on the js-side, and return a response from elm with an Id so you can map it to its corresponding response-object

#17

So about the frequently re-occurring ‘how to use Elm on a server’-question: It has to be said that Elm’s cousin Haskell has multiple full-fledged (i.e. large enough to have books written about them) webserver-frameworks (Yesod, Happstack, Scotty, Servant, interoptability between many solutions using WAI).

Of course, Haskell’s learning curve is slightly more cliff-like than Elm’s, because it contains more features and some peculiarities that still exist for historic reasons, and Haskell’s own way to do IO is a bit more difficult to learn than The Elm Architecture (That said there are implementations of TEA for Haskell)!

#18

There is server and server. To proxy requests while having access to secret env variables, Elm as is is just fine.

#19

Sure, there are some wrinkles in my Elm server (it is on the explicit side) but so far I have much better experience than I’d have with Haskell :slight_smile: YMMV obviously.