Do you have an opinion/advice on functional backends technologies?

Request Feedback
1

Hi,

My question is completely off-topic and unrelated to Elm per se of course, but I thought I’d ask Elm programmers for feedback still.

I’m kinda frustrated with my current tech stack and looking at alternatives.

I’ve been programming on and off for about a decade, as a self-learner.
I wouldn’t call myself a great programmer and work alone, so if I dig myself into a hole, no one’s gonna save me :wink:

Until recently, I resisted the notions of functional programming, maybe out of laziness or close-mindedness.

Anyways, I think I’m realizing I really enjoy the constraints of functional programming. An imperative language is just too powerful and makes it too easy to shoot oneself in the foot.

Imperative code did seem more appealing to me at first because I thought it made more sens to stay close what a CPU would do rather than introducing a huge layer of indirection.

I also like the testing benefits FP seems to provide: I always hated mocking, stubbing and having to rely on framework internals to do testing.

Static typing is a godsend once your project starts to get big. Also it enables dramatic tooling improvements. I like the idea of being able to refactor fearlessly knowing a compiler has my back. Snappy feedback is important for me too, I find waiting for stuff to happen frustrating.

I prefer one way of doing things (as in Python), rather than in many ways (as in Ruby).

So I very much enjoy Elm for all of that, and I think it may well represent all I want from a language. So I could see myself investing time into learning this language.

I’m thinking about going all-in with the functional paradigm, so looking at different technologies for the backend and I’m wondering if some of you have any feedback on that?

I gathered Richard Feldman (who made me curious about Elm via YouTube) was into Haskell at first, then switched to Rust. In a talk, he mentioned compile-time type safe SQL with a Haskell backend which definitely peaked my interest. I’d be interested to hear about a reason to switch to Rust from Haskell.

The Phoenix framework possibly looks like the best option for an FP backend. Problem is the lack of types from Elixir: I already know I don’t want to go that route.

So to sum up, the things that really click with me are:

  • fast feedback loop while developing
  • fearless refactoring
  • ideally good editor support
  • reduce runtime exceptions as much as possible (hello null pointers)
  • simple rather than easy, even if it means more “boilerplate”
  • a decent web + SQL ecosystem since my time is limited
  • I prefer writing SQL, but an ORM-like tool can be useful for productivity and expressivity (and encapsulation)
  • I don’t care about syntax

My experiences and what I like/didn’t like are:

  • ruby: great for scripts, poor refactoring and editor experience. Don’t like meta programming, too easy to overuse or badly use.
  • rails: great for RAD, but painful as the project grows and requirements change. Too much magic.
  • golang: great dev experience, hasn’t fixed the null pointer pb. Web + SQL ecosystem isn’t that great IMO
  • java/spring: Dependency injection breaks the benefits of the static typing IMO. Great tooling/editor support. Slow feedback is a major pain (jvm startup). Also most java code feels over-engineered to me (see the spring-boot-security architecture for instance, crazy and a bad design gets in my way). Annotations would be useful if not overused, makes the code execution path unclear.
  • angular/typescript: dynamic features break the static compilation advantages again. Very complex architecture and not a fan of the testing story.
  • vue/react: I wasn’t convinced. Although Elm inspired, I think introducing a system like a redux store does not feel right on those systems, quite a lot of boiler plate with no garanties, easy to make mistakes and introduce super weird bugs and bad refactoring abilities.
  • java/quarkus: maybe will have a good dev experience. Could be interesting if coupled with Kotlin (which seems to be a better Java with no downsides)
  • clojure: REPL driven development is an interesting concept. The lack of static typing hurts refactoring and tooling though. And also makes introducing runtime errors much easier.
  • nodejs+typescript: interesting combo and SQL tooling (though a little rough). Poor null handling (Swift or Rust seem to do a much better job at this)
  • svelte: looks very interesting but project is little young. Probably Elm’s only decent alternative for me.

So there you have it :wink:

Usually, I tend to just dive in and make my opinion but this time I think the FP landscape is too large for me to do that and I need a little guidance.

So I hope some of you went through the same process and have an opinion on the matter. I’ve looked briefly at Gleam and Elchemy. Both look interesting but a little scary to adopt since in such a niche.

Thanks for the feedback and have a great day

5 Likes
2

Its still a work in progress, but if you like Elm you might like to try elm-serverless 4.0.0

Here is a project that I am working on at the moment, that has 2 HTTP APIs implemented in Elm:

I’m sorry to say that I do not have a SQL implementation for it yet - the main reason being that so far I have only used it with a no-SQL database, DynamoDB. At some point I would like to look into what serverless SQL database offering are available on AWS (I’m hoping for a pay-on-demand Postgres), and then figure out how to do SQL in Elm to make use of that. The data model I am currently working with is getting more relational, so DynamoDB is getting harder to use.

For something better established, I think Haskell is going to be your best bet - The Elm compiler makes a good study in writing a significant sized project in Haskell.

If you use Haskell as a backend there is some support available for linking together Elm and Haskell types: haskell-to-elm: Generate Elm types and JSON encoders and decoders from Haskell types. One thing I really like about elm-serverless is being able to define the API data model once as common code shared between front and back ends written in Elm.

Or maybe OCaml? been years and years since I touched that…

1 Like
3

I’ve been doing Elm frontend/Phoenix backend for a while and I generally like it. You can get some decent type safety features by using dialyzer/dialyxir. It’s nowhere near as good as with Elm, but it’s definitely a few strong steps closer. It’s also possible in some cases to have a codebase that mixes Elixir and Gleam code since they both run on BEAM.

I’m not sure this will happen, but I’m hoping that over time it’ll get easier and easier to write more and more Elixir code in Gleam. Time will tell. I haven’t tried Gleam myself beyond watching some videos but I’m hoping. I would love to try doing some integrating of Gleam into my Elixir projects but so far it seems to me like the Gleam/Elixir interop story is still a little more complicated than I would like.

I mostly try to avoid macros in Elixir for the same reason you avoid metaprogramming in Ruby. Too powerful; too easy to make a mess.

Writing tests in Phoenix is one of the biggest pros for me. I find it much easier than when I worked in Rails. For me, writing tests in Rails felt like a chore I had to do. In Phoenix, it often feels like the easiest way to find out if something works, which I love. It’s hard to explain why, honestly, but that’s been my experience.

4

I’ve been doing some work recently in F#. I have issues with F# on a number of details, but I have been able to do the same sort of pure type-driven development as in Elm much of the time. There is more of the C# world than I would like that bubbles up at times, but it works and the foundation looks to be relatively well supported. You can use Fable and Elmish to do your front end work in F# as well if you want to share types across the front and back end though the code is a bit messier than Elm and the error messages not as good. I will say, however, that it’s pretty cool having one watcher rebuilding the server code while another watcher rebuilds the client code.

2 Likes
5

For hobby stuff I’ve really been enjoying Lamdera (proprietary, alpha), which provides a way to do frontend and backend in one elm codebase, with persistent data on the backend and migrations.

It’s so nice that if it were mature and had a couple more features I’d be using it for anything that doesn’t need a real database.

For a larger project I went down the haskell / Yesod path several years ago.

First of all the bad stuff: Although the book makes it seem as though “anyone can set up a yesod server”, you need to know haskell quite well to be able to do anything that’s not handed to you on a plate. Expect to dig deep into the code as the book and docs are outdated and missing key details. Libraries are re-exported everywhere, so it takes alot of browsing to find where a function might have come from.

I had a couple of haskell projects under my belt, but even so I found it was quite a learning curve as there is much type-level and template magic that you need to understand in order to change any default behaviour. In the beginning the phrase “just modify the default implementation” actually meant “a week of reading and experimentation to figure out how the types fit together and all the hidden side effects”. There are still chunks of the code that I cannot write type signatures for. There are areas that I’d love to refactor, however the pieces cannot be pulled out into their own functions without type hell. Yesod definitely won’t let you write Haskell like it’s Elm.

Having said that, Yesod has been rock solid, fast and very easy to extend the site once you understand the hidden magic. The Persistent database story has been just great with Postgres behind it.

I’ve also been able to replace most of the ‘Shakespeare’ html templating with Elm, so now I have one codebase to deploy that covers everything from database, static content, server-side generated content, to several elm applications and an api server.

Now that I’ve put all this effort into it, I’m quite happy with it, but I think that’s probably Stockholm syndrome. Next time I’d probably switch to a simpler Haskell framework that allows Persistent as the backend.

3 Likes
6

Thanks for the suggestion. I’m hesitant about serverless. I could see the appeal, but I don’t think it would fit my use case. I’ll definitely keep your project in mind though, looks very interesting :slight_smile:

The Haskell to Elm type sharing looks very appealing though!

Ocaml looks interesting at a quick glance, but I’m wondering how active the language is? I note that there is no builtin unicode support, so looks like a bad sign.

7

Thanks. I looked into what BEAM can offer a developer, and the platform does seem appealing.

I’ll have to try out the Elixir/dialyzer combo though I understood the Elixir support is subpar, rather it works best with Erlang.

I tend not to like transpilation, preferring just using the underlying language (thinking about CoffeScript, GopherJS and so on…). Elm is an exception to that though, since it brings a whole now set of benefits. And TypeScript which brings some real tooling improvement thus makes you more productive.

All that to say that I’d rather go with straight Erlang, I’m not sure yet what benefits Elixir brings appart from subjectively “better” syntax, and meta programming.

Projects like Phoenix do make an appealing case for using Elixir so I’ll have to dig into this. Not sure indeed how one could mix different BEAM languages in the same project.

8

Thanks, I’ll look into F#.

I forgot to mention it but I did look at C# and the .Net core framework. Ecosystem looks pretty good but not enough experience with it to have an opinion.

Are you use F# with .Net Core?

Regarding sharing types, I’ve been thinking that a technology like gRPC could help. Anyone has experience with that?

9

Thanks, I wouldn’t use a lock-in solution like Lamdera. Makes me think of meteorjs which presented the same value but did not succeed probably because most devs feel this way.

First of all the bad stuff: Although the book makes it seem as though “anyone can set up a yesod server”,

What book are you referring to?

I did gather that Haskell is much harder to learn than Elm, so nice feedback to have. I should have mentioned that good documentation is important.

So if a project has poor documentation, I probably won’t give it a chance since my time is too limited.

Could you give a little example?

I’d like to hear more if you have the time :slight_smile:

–

Damn sorry for all that noise (4 posts in a row), I should have used quoting sooner!

10

Yes, its still too much of a work in progress to be able to recommend using it outside of hobby/personal projects at the moment. The name ‘serverless’ is a little misleading, you can run it outside of the cloud with sls offline - in which case its nothing more than an HTTP service running on Node. I should probably add and document better support for running outside of the serverless framework altogether.

The appeal of serverless, as in AWS Lambda functions, is that they can be astonishingly cheap to run with no effort spent on managing the servers.

11

The main documentation effort for Yesod is in the Yesod Book. It’s pretty good at explaining what’s going on, but doesn’t replace good, complete library documentation. Yesod’s docs are ok for alot of the basic functionality, but when you get in the weeds there are more questions than answers.

There are still chunks of the code that I cannot write type signatures for.
Could you give a little example?

Haskell has something that Elm does not: type classes. This allows you to have ‘generic’ versions of types, however for type signatures means you sometimes have to give the compiler some extra information on constraints. To take an example from the scaffolded Yesod project, let’s say you need to create a function for authenticating which takes a Credential, queries the database, and returns an authentication result.

authenticate :: Creds App -> m (AuthenticationResult App)

Here App hides a whole bunch of underlying functionality and data about the state of the server and database, and m means this formula will return whatever monad context that the authentication is running in.

For this to compile, however, you need to constrain the types so that the compiler is happy, but also so that all the possible context that you might be authenticating in are covered. Something like:

authenticate :: (MonadHandler m, HandlerSite m ~ App) => Creds App -> m (AuthenticationResult App)

So this might compile locally, but now other parts don’t compile because their type signatures require some other constraint - for example the test suite uses alternative mock versions of App and Handler types.

“The Persistent database story has been just great with Postgres behind it.”
I’d like to hear more if you have the time

Lay out your model in a structured text file that Template haskell uses to run code for you to do any database setup and migrations. Then insert, delete, query using an api that’s kind of Dict-like if you squint the right way, and adds some magic so it’ll ‘do the right thing’ based on the declared type. So for example:

getUsersR = do
     usersE <- runDB $ selectList [] [] :: Handler [Entity Users]

gives you an Either holding a list of all the users based on the declared type. Any trivial migrations are handled by Persistent, which is nice during development.

[Edit: clarity and formatting]

12

I’ve thought about this a lot and come to the conclusion that it’s either F# or Haskell which offers the closest experience to Elm.

Haskell has a lot of very complex libraries which makes the language really hard. Instead, you can instead go with a serverless solution using Hal (hal: A runtime environment for Haskell applications running on AWS Lambda.) which is used at Nike. With that you can use either API gateway (Rest apis) or AppSync (graphql api). The benefit of this is multiple:

  • By using haskell serverless, you can avoid a lot of the complex parts of the haskell ecosystem. Most Haskell backends have to employ a very complex ReaderT monad with all kinds of resource managers and other things. Basically, because monolith Haskell needs a swiss army knife of type-safe features, it gets very complex and hard to learn. However, a single lambda won’t need anything complex beyond the IO monad so you get to stick with the nice parts of Haskell that we know, love and learn in university plus also reduce the complexity of the language.
  • Haskell has a notoriously high compilation time. Going serverless will greatly reduce this since you just compile a single lambda.
  • Haskell has missing libraries. With a serverless solution you can add javascript lambdas or python lambdas if needed. Things like setting up OAuth authentication was a PITA in Haskell but can instead be handled by AppSync or API gateway. You reduce some of the big risks with Haskell this way.
  • Haskell’s IDE works really poorly in large projects but very good for small lambdas.
  • You can use hasql (GitHub - nikita-volkov/hasql: Performant PostgreSQL driver with a flexible mapping API) for type safe queries. Much simpler than many Database tools.
  • Haskell has a very fast cold start time which makes it a good fit for lambdas in general.
  • dhall-lang for safe configuration management. This can be used to create a type safe connection between AWS API gateway and your haskell lambda for example.

The drawback is of course the complexity that comes with a serverless environment and using aws.

Another alternative is to go with F#. F# has a very good ecosystem since you can interop with C# if any library is missing. It also has very very good tooling and similarly to elm focuses on a culture of ease-of-use. Some neat libraries it features are F# Graphql server, sql providers (for type safe queries), json providers and suave. F# is less suited (would still probably work fine tho I think) for serverless lambdas due to much higher cold starts. However, you can just have a monolith and host it on Heroku instead. I would say this is a better alternative if you just want to get your idea out.

6 Likes
13

While Rust is not pure functional, it meets most of your requirements from here:

  • Fearless refactoring - yes. Rust’s basic type system is very much like Elm’s - primitives, ADTs, Maybe & Result, no null pointers, etc. On top of that, there’s a Trait system that I understand is much like Haskell’s typeclasses. Finally, even though there’s not garbage collection as such, the “manual” memory management is checked at compile time and programs won’t compile if it’s not OK.

  • Good editor support - yes. Rust has good support in all the editors I know of.

  • Reduce runtime exceptions - yes. First, no null pointers. Second, no segfaults, buffer overruns, and everything else. Within the language, Rust doesn’t have exceptions as such. It does have “panic”, which in the naive case just crashes the program, but you can catch and handle panics, so it can get ugly if you want it to, but that’s not normative.

  • Decent web + SQL ecosystem - yes. For Web, Warp is the new hotness, and there are several other more mature frameworks if you want it. For SQL, sqlx is the new hotness, and there are several other frameworks as well – Diesel is the more ORM-like system.

  • Prefer writing SQL - I believe sqlx is more like that than most ORMy things, but I haven’t dived into it yet. (Hopefully this month!)

  • Don’t care about syntax - that’s good, because Rust’s syntax is an attrocity.

  • Fast feedback loop while developing - sorta. The compiler is as strict as Elm’s, so in that sense you’re not waiting for runtime errors to figure out if things work or not. However, the compiler is Way Slower than Elm’s, so you wait seconds to minutes on every build.

Finally, there’s a big cognitive hurdle in learning and internalizing and developing intuition for Rust’s novel memory management system. I think it’s worth it just for the journey to another universe, but it’s not the fastest language to get your first web app running in.

9 Likes
14

I will suffix that that I regularly ragequit Rust, but then crawl back because the other options are worse. But it’s not a panacea.

3 Likes
15

IHP might be a good choice if you’d like to invest into its community https://twitter.com/digitallyinduce/status/1358670199197679617?s=21

1 Like
16

We are using the .NET Core framework together with Giraffe and Thoth. In particular, we are using Thoth for JSON encoding and decoding. It can actually do that automatically, but Elm has trained me to write my own encoders and decoders and therefore to take control of/be aware of the actual JSON encoding choices. But the code for those works cross-platform (with a conditional include and some careful project structuring to keep tooling happy) meaning that we can use the same types on either side of the wire where it makes sense to do so.

Being built on top of .NET means that it has access to all of the standard C# libraries, but thereby also means that you can end up interacting with C# libraries that weren’t built for a pure functional world. I had to spend longer than I would have liked wrapping the AWS Dynamo .NET libraries (good to have available) to get them to report errors via the Result type rather than via exceptions. (AggregateException did not endear exceptions to me.) Less frustrating but something to watch out for is that Giraffe builds on .NETs HttpContext object and I believe uses it mutably. Maybe it spawns new objects at each mutation, but I wouldn’t count on it without specific documentation saying it does so.

I find some of the syntax overly noisy. This mostly comes from C#.

I find some of the code and type structuring ugly. This, I believe, mostly comes from ML.

F# has convinced me that while white space sensitivity can make for pretty code, it can also make for code that is easy to foul up when editing. For example, F# doesn’t require a divider token between list elements if they are put on separate lines. That’s pretty until you do something that causes one line to shift to the left or the right and then to allow an expression to run on from one line to the next because the latter is now indented relative to the former.

But the overall experience is a lot like working in Elm and on what is admittedly a much smaller project than my last Elm project, it has generally enabled the same sort of fearless refactoring with the caveat that because it is more imperative friendly it doesn’t make quite as many guarantees.

17

Indeed. For the moment my main devop tool is “dokku” (my needs are modest), which reduces maintenance quite a bit. Combined with load balancing, I found it quite optimal before having to bite the bullet and having to adopt kubernetes.

I’ll definitely keep your project in mind though!

@Brendan thanks for the detailed explanation. I don’t understand the details of course but I think I get the gist of it. Yesod 1.6 came out in 2018 if I’m not mistaken, but the book mentions being still a work in progress so I’m not sure I feel to good about that.

Thanks for bringing it up again. F# does seem to haven some very interesting features and tooling. The npm integration mentioned here is making me curious.

I think you’re right. I dabbled with rust a while ago, it’ll definitely require a bit of effort to understand the language fully and get productive. I also had a look at Diesel a couple of years ago or so, it didn’t really appeal to me at the time, I think due to the fact that it felt like straying too much from the actual SQL. I’ll take a look at sqlx. At a quick glance I’m puzzled at how the SQL type checking actually works. Good to know about Warp too thanks.

Thanks, I don’t think I would have found easily about this Haskell framework. Here’s a link to its guide if anyone’s interested: IHP Guide

Great feedback, I appreciate it. I understand that F# makes the same compromise than Clojure, letting object oriented stuff sneak in if necessary. Might not be a bad call, I could see the advantages. I’ll definitely try F#, it wasn’t on my radar at all.

1 Like
18

I’m working with the Daml smart contract platform, it’s a very good match to Elm.

See: Daml Programming Language

And my blog post about this combination: https://link.medium.com/mAzk7raGEeb

19

I think Elm has some potential as a cloud language, more so than as a general purpose back-end language. So-called ‘Lambda’ functions on AWS interact with the rest of the AWS environment almost exclusively over HTTP - which means that the familiar elm/http package is already available to work with them.

As a general purpose back-end language Elm lacks support for concurrency. So you cannot write a cache which is shared amonst worker threads processing incoming requests. High perf servers are sometimes built as pipelines with a sequence of stages processed by worker threads, passing work down a pipeline; with #threads = #cpucores - 1 (to leave 1 for the OS). That kind of producer-consumer concurrency pattern is necessary there.

But for the cloud, i think Elm can work nicely. One of its attractive features as a cloud language is that with a stricter compiler, less bugs make it through to being deployed in the cloud - It is already hard enough making reliable cloud apps because they are distributed systems. I think the way in which Elm returns errors rather than simply failing generally forces the programmer into thinking about all the error cases up-front. Deploying to the cloud is very time consuming, unlike the quick hot-deploy cycles that you set up when working on the UI. Its almost like the old days when you went to make a cup of coffee while the compiler was running. Getting the mistakes out at compile time is a big time saver here, and the most commonly used languages, JS and Python, do not score highly on this.

I’ve been generating a few Elm stubs for AWS services:

https://package.elm-lang.org/packages/the-sett/elm-aws-cognito/latest/
https://package.elm-lang.org/packages/the-sett/elm-aws-elastic-containers/latest/

Some AWS services work fine through the stubs - the AWS SDK is just a thin stub to an HTTP API call in the majority of cases. For some, like say DynamoDB, the SDK builds a significant software layer over a lower level API. So far, I used DyanmoDB through Elm ports and the AWS SDK, rather than try and work with its lower level API. Potentially a nice Elm package could be built on the lower level API though…

One thing I have found with elm-serverless is that when you have longer chains of side effects, turning every step into a Msg can be tedious. Longer chains of side effects are common - maybe your code reads from a queue, then a database table, then invokes some other service over HTTP, then writes to a database, then puts a message on a queue. That is not untypical of the kind of sequence of processing that Lambda functions are used for. Once I’ve got my current project working, I want to do a re-write of elm-serverless, along the lines of elm-procedure 1.1.0.

Sorry, not a recommendation, just some thoughts on Elm on the back end and a bit of an update on Elm serverless on AWS.

2 Likes
20

In my experience Lambda functions are long chains of promises, I wouldn’t have a clue how to do that in Elm, except through Msgs, and not sure that would improve understandability/readability.

But haven’t tried Elm on the backend yet, agree not having to debug would be an enormous leap forward from using JavaScript.