Announcing Elm Codegen!

:wave: Hello!

I’m super excited to announce elm-codegen, an Elm package and CLI tool for generating Elm code!

My goal is to make it easy to write maintainable code generation for Elm.

Basically I wanted something that was easier to write than that string template you have lying around. :smirk: And more maintainable than juggling a raw AST.

Here’s a simple example!

This code

Elm.declaration "anExample"
    (Elm.record
        [ ("name", Elm.string "a fancy string!")
        , ("fancy", Elm.bool True)
        , ("theTime", Gen.Time.millisToPosix 0)
        ]
    )

will generate the following:

import Time

anExample : { name : String, fancy : Bool, theTime : Time.Posix }
anExample =
    { name = "a fancy string!"
    , fancy = True
    , theTime = Time.millisToPosix 0 
    }

Let’s break down what this tool does for you in order to make generating code simpler.

  1. Automatic imports — Import declarations are calculated. If you use a value from another module, it will know and write your import declarations accordingly.
  2. Built in type inference — The types for your generated code are inferred. This means generated things can figure out their own type signatures!
  3. Use existing packages easily — For generating code that uses a specific library such as elm-ui, the elm-codegen CLI can create some Elm code to help you out. In the above example, we’re using generated bindings to the elm/time package.

Using Existing Packages

Let me elaborate on the 3rd point for a moment because this turned out to be one of the most interesting aspects for me.

Using the elm-codegen CLI, you can install bindings to any package from the Elm universe.

For example — running elm-codegen install elm/html will create the following files —

  • Gen/Html.elm
  • Gen/Html/Attributes.elm

These will help us generate calls to Html. What does that look like? Well, we can now write something like this:

import Gen.Html
import Gen.Attributes

div : Elm.Expression
div =
    Gen.Html.div
        [ Gen.Html.Attributes.class
            "hello"
        ]
        [ Gen.Html.text
            "Hello world!"
        ]

Which will generate

Html.div
    [ Html.Attributes.class "hello" ]
    [ Html.text "Hello world!" ]

And would automatically include imports for import Html and import Html.Attributes

Crazy, right?

The generated helpers also include a number of other ways it can assist you, but for me, this basic mirroring of how you would interact with a library normally makes things very intuitive in a lot of cases.

The Guides

There is a very succinct guide to get people started:

Also, major thanks to Leonardo Taglialegne aka @miniBill, you’ve been a massive help in developing this tool from the very beginning, I’m incredibly grateful :pray:

Live, at Strange Loop!

I’m going to be giving a talk on this library at Strange Loop on this and a bunch of related work that is built on top of it (that will hopefully be ready for release soon!)

We use this at Vendr.com

We have roughly 300k lines of Elm code at Vendr.com, a wonderful, talented Elm team, and use Elm CodeGen to power our new GraphQL library (which I will be posting about very soon.)

I also have some fun stuff in the works with regards to this and elm-ui!

Happy to answer any questions. Lots more to come!

58 Likes

I will summarise the question and answer I had on Slack so that it is preserved here for posterity:

Q: Does the package helper work for all published Elm packages?
A: Yes it does, and has been verified against all the Elm docs JSON.

Very nice feature, well done with that! Until now I was using my elm-syntax-dsl but manually writing generators against the packages I was using and this could be a real time saver.

3 Likes

There is something else I am curious about… the type checker? I seem to recall Evan mentioning that to make it efficient in the compiler mutable state is needed, which I guess is possible in Haskell but not in Elm. Is that something you found when porting the type checker from the compiler into Elm?

1 Like

Yeah! One of the final tests I did was elm-codegen install for every package and confirmed that the generated bindings compiled. If people are interested in a cache of all the docs.json files for the package website, I also created this repo: GitHub - mdgriffith/elm-all-docs: A cache of all the docs.json files on the Elm Package website

Good question! Right, as I understand it, any pure language is going to run into some challenging performance issues during type unification.

So far I haven’t actually run into this in the wild though. So, maybe it just matters less for this project than for a full language?

My thinking is

  1. elm-codegen is doing less typechecking work than a proper language. (e.g. we’re not directly resolving values from an imported module)
  2. In much the same way you can elm-codegen install {package}, you can also install local Elm files and it will generate some code to help you call that local file. This is an effective way to reduce load on the type inference in elm-codegen because those “helper files” effectively have all their type inference work for them already baked in.

If people do end up with a challenging performance situation, there are a few options to explore.

  1. I imagine my existing code could be sped up via normal means.
  2. GitHub - mdgriffith/elm-optimize-level-2
  3. Add the ability to turn off type inference in the library

I imagine there are other things that could be done too.

So, if we notice performance being an issue, please file an issue on the repo! Happy to look into it further.

3 Likes

I don’t want to be rude, but I don’t get the examples… they are very abstract to me. Maybe you could give a more concrete/real-world example?

Maybe I’m not the kind of elm-user this is intended for. The first thing that I hoped I could solve is to generate a List of custom type values:

type Animal = Dog | Cat | Snake

gen-magic...

generates:

animals : List Animal
animals = [ Dog, Cat, Snake ]

Could this tool be used for that kind of code gen, or am I totally on the wrong path here?

2 Likes

Not rude at all! I actually love these kinds of questions. And code generation is actually kinda mind bending, even if there is a nice library to support it.

If you wanted to generate that custom type, you would write this

module Main exposing (main)

import Browser
import Elm
import Elm.ToString
import Html exposing (Html, button, div, text)
import Html.Events exposing (onClick)


myElmStuff =
    Elm.customType "Animal"
        [ Elm.variant "Dog"
        , Elm.variant "Cat"
        , Elm.variant "Snake"
        ]


main =
    let
        rendered =
            Elm.ToString.declaration
                myElmStuff
    in
    Html.pre []
        [ Html.text rendered.body
        ]

Which you can check out in this ellie example. And here are the docs where it talks about that.

The easiest way to get started with this library is to play around with stuff purely in Elm, like the Ellie example is doing. Honestly I should add a bunch of Ellies to get people started.

The only “magic” is really the elm-codegen CLI tool, but you can skip that part while getting familiar with the library.

Let me know if you have more questions, I’m happy to discuss. Your perspective is helpful!

1 Like

The way I understood the question was not “how codegen the type Animal?” but instead I think the question was “how do I codegen the constant animals with the given type Animal?”

Hope I don’t misunderstand stuff here…!

2 Likes

Yes. How to generate a list of variants was my question.

Here is my ellie so far. It generates a list of strings, because I don’t know how to turn the Variant-Type into an Expression. See line 30 (commented).

Not rude at all! I actually love these kinds of questions. And code generation is actually kinda mind bending, even if there is a nice library to support it.

@mdgriffith It looks lilke a very powerfull tool, once you understand how to use it, which I don’t yet :wink:

1 Like

You want to check out Elm.value!

I made the changes here:
https://ellie-app.com/jnC3DSLD8LKa1

1 Like

Thank you very much!!! :tada: :pray:

1 Like

No problem! :tada:

You’ve made me realize I should probably extend this guide: elm-codegen/WritingAGenerator.md at main · mdgriffith/elm-codegen · GitHub

To cover some more real examples.

5 Likes

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.