Exploration for Server-side Rendering

#1

I raised this on Slack. Moving the conversation here, since regardless of the outcome I suspect there is value in capturing ideas in a place where they can be referenced in the future.

In the local JavaScript community here in Melbourne, Australia, the #1 thing that people cite as a reason for holding off exploring Elm in my conversations with them is lack of support for server-side rendering. If the focus of the core development team is on other important things right now, I wonder if there is room for an elm-explorations initiative to establish a community-supported model for rendering Elm views on the server and then having that pre-rendered HTML DOM “adopted” (or “hydrated”) by the Elm runtime in the browser.

Note: Static server-side rendering (generating a non-interactive web page from Elm views on the server) is a simpler problem. I’m told elm-pages does this very well, which is great. But a lot of potential Elm adopters really need their interactive Elm app to be able to serve its initial view pre-rendered by the server, and then have it become interactive on the client-side.

Reasons for wanting this:

  • Time to first paint (which impacts SEO)
  • Accessibility with JavaScript disabled (which impacts SEO, especially on non-Google search engines)

The problem is not a trivial one to solve. Knowing at what point an Elm application has “settled” (e.g. fetched any data required to render a useful initial state), and is ready to be sent to the client, and also capturing that server-side state and attaching it (and the server-rendered HTML DOM) to the client-side runtime are both tricky things to do well. Elm’s immutable data design makes it perhaps uniquely suited to doing this reliably, however!

I believe there have been some experiments done with SSR for Elm by others, and I’d love to collect some links to those on this thread, as well as brief summaries of what those experiments were able to achieve (and perhaps what they weren’t). If it should happen that a 90% solution is already out there and it just needs some community support around it in the form of documentation, blogging, conference talks, etc., I’d be very excited to help out with that (rather than with code, which we all know is the easy part).

A couple of previous threads on this topic:

Explorations in Server Side Rendering
Server-side rendering in Elm 0.19

10 Likes
#2

I think https://github.com/rogeriochaves/spades provide already server side rendering don’t know the methodology if is what should be different but in terms of server side rendering as a feature should be there already.

#3

In the project he posted about in Explorations in Server Side Rendering, @ktosiek cites Spades as a source of inspiration. I’m curious if @ktosiek would be available to share what unsolved problems were left by Spades that led him to start his exploration?

#4

Sure!
The things I couldn’t find in Spades when I’ve made that demo were:

  1. model (de)serialization,
  2. waiting for the view to settle.

My demo can wait for XHR and I think it should work with synchronous Cmds (like getting a random number or current time).
Spades could render a simple shell for your app, but not much more. I don’t know the current state of Spades.

Edit: looks like Spades only gives the app one “tick”, and then returns whatever got rendered: https://github.com/rogeriochaves/spades/blob/1c87b2ab7bfb2c865a43022268bb890df34b03de/boilerplate/server.js#L69

2 Likes
#5

What if an app depends on some information that is only available on the client to do the rendering? Like checking the window width and height instead of using media queries (this is what you do with elm-ui for example), devicePixelRatio to decide which size to use for images, etc. As far as I know, there is no way you can get this info on the server, so SSR might not be possible depending on how you write your code – which means (correct me if I’m wrong) that it’s not possible to implement a generic solution that would work for every case (and this problem is not specific to Elm).

#6

I believe you’re correct: there are many subtle trade-offs and limitations like this that must be dealt with when implementing SSR.

From this article on React SSR:

  1. Make sure you don’t reference window or document

Your React components will now be rendered by your node backend. And your node backend doesn’t have the window or document global variable. That could lead to errors like this on server side:

window is not defined

If you get a similar error, you can put the code referencing the missing variable in an if statement like this:

if (typeof(window) !== "undefined") {
    window.localStorage = ...
}

So an SSR approach for Elm would need to deal with the fact that some effect handlers would not be supported in the pre-rendering environment.

As to the specific question of elm-ui, if I were implementing an app that I wanted to have SSR, I would be happy to accept the constraint that the content’s layout must work without JavaScript (perhaps doing progressive enhancement with JS for non-essential features).

In an ideal world, where Elm had first-class support for SSR, I would imagine that effect handlers that depended on the window object might support returning a result like NotSupportedOnServer, which you could handle as appropriate in your app.

#7

Over the last year, I’ve been exploring the different approaches that people have taken to achieve SSR with Elm.

Rather than using JSDOM or Puppeteer, @eeue56’s elm-static-html-lib took a JavaScript object, decoded it (from a decode function referenced in Main) in Elm and then was passed into the Main’s view function to render as a HTML on the server.

However that library is now deprecated as of 0.19, although it was forked by Daniel Wehner: https://github.com/dawehner/elm-static-html-lib.

The tradeoff with the above, was that the Model needed to be identical to the JavaScript object.

With all of the different approaches, I haven’t seen a rehydration technique similar to what was used by @ktosiek. @ktosiek, I think if you can further develop your elm-ssr-demo exploration, so that it encompasses tasks and commands, that would be awesome.

Also as a side note, the elm-ssr-demo isn’t working for me now we have the 0.19.1 update – I’m getting a “Mangling failed on regexp” error.

1 Like
#8

In the local JavaScript community here in Melbourne, Australia, the #1 thing that people cite as a reason for holding off exploring Elm in my conversations with them is lack of support for server-side rendering.

Do they ever say why SSR is so important?

If the concern is SEO, I know there have been discussions around rendering your app in puppeteer and serving up that response.

If the concern is time to first render, wouldn’t Elm’s tiny build size be of benefit over other frameworks? Also wouldn’t research around code splitting be of more use as you could get your build size to be even smaller?

#9

From my point of view is more like we don’t have a way documented in the elm documentation.

#10

See “reasons for wanting this” in my original post above. In short: performance, SEO and accessibility.

Spinning up a full (headless) browser on the server is needlessly resource-intensive, and not especially fast unless you’re maintaining a fleet of “hot” browsers waiting to render things.

Yes, but it’s a case of “why not both?” Rendering with Elm is faster than React (say), and doing the initial render on the server is faster than doing it on the client. Combine the two for maximum speed.

Again, these are all beneficial performance measures. One does not preclude the usefulness of another.

That said, there would be value in measuring their relative benefits in real-world scenarios to see which should be the priority if people need to choose one to focus efforts on.

1 Like
#11

I can vouch that for this approach, it is a very hard problem to tackle. Securing and sandboxing the browsers (because they are running what could potentially be malicious user input, and the last thing you want is a compromised server), and doing the proper queueing and resource balancing is a very hard problem. Chromium and puppeteer under load can have very unpredicable behavior.

At work we have something like this for rendering PDFs from web content, and it was a long and challenging project to get sufficiently right for production usage. And now on the maintenance stage, keeping up with the chromium upgrades and the OS upgrades of the servers is also a non-trivial amount of ongoing work.

YMMV, but I advise caution with this approach for any non-trivial use cases.

1 Like
#12

I believe stuff like prerender IO doesn’t have a hot browser doing the prerendering, while the request happens. Rather it is more like a cron job crawling the page and caching the pages periodically. You then have a server in front of the prerender io cache that merely serves the cached pages IF it detects you are a search engine bot. Regular users will be served the regular index.html which then starts loading the JS bundle.

I think it makes sense to categarize the prerender approaches into 1) solutions that crawl and cache periodically and 2) solutions that do actual SSR upon request.

For the latter I agree it would likely too slow to do it with a headless browser.

#13

Server-side rendering of dynamic (user- or query-specific) responses is the gold standard that the React community here looks for in alternatives.

1 Like
#14

One issue with static impressions of an Elm app is knowing when to do it - when to consider the page has settled. The one point it can be done deterministically is right after the init function; take the view that results from that first Model. Any other time could be non-deterministic since init could return a Cmd.batch and the order in which those hit the update cannot be guaranteed. Consider starting a timer and issuing an HTTP request, which will complete first?

There are SSR techniques that work with Elm already, I am trying to think of things it cannot do, and whether the Elm compiler or runtime could support them better.

Would having some event you can listen too, in order to know when init has completed (or even when update or view are being run) be something that needs to be added to the Elm runtime?

#15

A technique I’m using is to notify a port when all the necessary effects have been processed by update. Once the JS (running JSDOM in my case) receives that, it needs to know when the next view function has completed. Unfortunately there is no callback for that, so I set up a mutation observer and assume that the next mutation is the VirtualDOM done.

This works OK without too much manual work (basically a single out port), but feels a bit brittle. A callback after view rendering would be much nicer.

1 Like
#16

Yes, I think you are right, that is where it needs to go. Its not after init its after the first view.

Using a port to signal when all necessary effects are completed is one way to do it. Means each SSR framework needs to define which port to use. So I guess this could be another built-in that the Elm runtime could provide.

module SSR exposing (..)

{-| Once this command has been run the `onViewComplete` event 
will have the `settled` flag set to true.
-}
settled : Cmd msg
#17

I think always taking settled as being on the first view has an advanatage though - it does not allow any effects to be run. This means that SSR doesn’t have to worry about say Browser effects that won’t work right in that context.

For example, often one of the first things my SVG apps do is to get the window size or an element size, to help set up an SVG canvas that is 1:1 with the pixels. In an SSR context there is no window size to know, since that doesn’t come in with the GET request for the static page. I might still render some framework for the page, and then do the SVG drawing after rehydration.

This would save on the need for some effects having to change their APIs to report that they are not available in an SSR context.

#18

I’d say that’s an untenable restriction. It is very common for Elm apps to init into a “Loading” state, which then fetches content from the server. Only once that content is loaded and the view updated is the view content-complete, and it’s that view that we want to send from the server, so that search engines and clients without JavaScript can access the content.

If the Elm runtime were going to attempt to pick a sensible moment to declare the view “stable”, I’d say it would be the first view after all of the commands triggered by init have run, the resulting updates processed, and the first view after that rendered. But even that would be full of edge-cases.

Server-side rendering an empty shell with a loading spinner is not very useful.

@rupert the “SVG canvas” type of application you’re thinking of doesn’t strike me as a particularly good candidate for server-side rendering. One local business that depends heavily on SSR, for example, is a job search website, which has category pages that initially list the newest jobs in that category, but which provide a rich set of filtering controls that you can apply and have the search results update (with each filter state having its own bookmarkable/shareable URL). That company considers it vital to have those job listing pages (as well as the individual job pages) load very fast (because search engines rank fast-loading pages higher), with complete accessibility to search engines and people with disabilities.

#19

As far as I see, there are 2 things that need to happen:

  1. A way to do the server sider render
  2. A new way to initialize the Elm app

The simplest model that comes to my mind for the server-side render is a Task that produces a String that is the rendered html. Alternatively it could produce the model and have the conversion to String happen automatically. All the commands that end up in init would have to be converted to a chain of tasks in order for this to work. I’m expecting that most of the init code could be converted to a Task.

The initialization part is a little bit trickier. I cannot figure out how to do it without adding yet another parameter to the Program type. In essence, the init needs to receive a new data type that would represent the payload to a message that would put the initial model in the “after the initial update” state.

2 Likes
#20

In my demo I’m mocking requestAnimationFrame to know when the rendering is over. Maybe that would work for you too? You could wait for the “settled” event from your update, and then run all pending RAF callbacks.

1 Like