Decoder generator and imports cleanup tool

[dkodaj]

I’ve written two small tools to help clean up code / automate the mindless parts of coding:

Decoder generator
Generates JSON decoders and encoders from type definitions.

Imports cleanup tool
Removes unused imports and makes “exposing (…)” imports explicit.

Just copy/paste your code and press the button.

This project may have benefitted from elm-ast or elm-parser, but I was pretty much done when I discovered those.

Feedback welcome!

[joelq]

I like the look and feel of it! I threw a bunch of types at the decoder generator to see what it could handle . I’m impressed !

It handles:

• Records
• Tuples
• Dicts
• Lists
• Nested structures like a list of tuples
• Sum types (e.g. type Color = Red | Green | Blue)

I’ve noticed is struggles with product types like type Email = Email String. If I were writing this by hand I’d implement the decoder as this one-liner:

decodeEmail =
  Dec.map Email Dec.string

but the generator attempts to decode it using a huge nested case and multiple calls to decodeValue

decodeEmail =
   let
      recover xs =
         case xs of
            a0::bs->
               case decodeValue Dec.string a0 of
                  Ok "Email"->
                     case bs of
                        a1::cs ->
                              case decodeValue Dec.string a1 of
                                 Ok a1_->
                                    Dec.succeed <| Email a1_
                                 Err err->
                                    Dec.fail err
                        _->
                           Dec.fail <| "Invalid fields for constructor Email: " ++ toString bs
                  Ok other->
                     Dec.fail <| "Invalid constructor field found: " ++ other
                  Err err->
                     Dec.fail err
            _->
               Dec.fail "Invalid JSON input: empty list"
   in
      Dec.list Dec.value |> andThen recover

[dkodaj]

Txx for the quick feedback!

This is ugly indeed, I’ll fix it up…

[rupert]

Removes unused imports and makes “exposing (…)” imports explicit.

Nice work.

I would find this more usable if it were available as a command line tool, it works nicely but not realistic to cope paste every file in a large project.

[dkodaj]

I’ve noticed is struggles with product types like type Email = Email String

This is fixed now.

[dkodaj]

I would find this more usable if it were available as a command line tool, it works nicely but not realistic to cope paste every file in a large project.

I agree. I’ll try to do that if the copy/paste version doesn’t uncover any obvious bugs.

[joelq]

I’ve noticed is struggles with product types like type Email = Email String

This is fixed now.

That was quick! I notice it works for multi-argument product types too

I figured I’d throw something a bit more complex at it:

type Suit = Hearts | Spades
type Rank = Ace | King | Queen

type Card = PlayingCard Suit Rank

It’s able to correctly generate decoders for this!

Trying something fancier, I notice it starts to breakdown when combining sum and product types. When I extended the example above to add jokers:

type Suit = Hearts | Spades
type Rank = Ace | King | Queen

type Card
  = PlayingCard Suit Rank
  | Joker

then it’s back to generating the deeply nested decodeValue stuff.

[Chadtech]

Pretty cool! I am always using elm-analyse to find unused imports, but I can never quite get rid of all of them, so having an automatic tool would be super useful for me. Thanks for working on this.

However, I did apply the unused import tool to four big modules of mine, and I had a lot of problems:

0 It would delete this import: import Data.Id exposing(Id(Id, New)), where the module contained…

type Id
    = Id String
    | New

1 I had an import that was like import Data.A as A exposing (B), and I had functions that used B in both aliased and unaliased forms like String -> A.B -> ... and String -> B and it removed the import for B entirely

2 I import a union type and expose many constructors like import Tracking exposing (Event(A,B,C)) and it would format that to import Tracking exposing (A, B, C)

[dkodaj]

Thanks, these are very helpful.

I’ll need to work on handling exposed types. That should take care of issues 0 and 2.

I haven’t been able to reproduce issue 1. The following yields import Data.A as A exposing (B):

import Data.A as A exposing (B)

myFunc1: String -> A.B -> Int
myFunc1 a b =
  5

myFunc2: String -> B
myFunc2 a =
  b

[dkodaj]

type Card
  = PlayingCard Suit Rank
  | Joker

then it’s back to generating the deeply nested decodeValue stuff.

This should work now. The result is the following:

decodeCard =
   Dec.field "Constructor" Dec.string |> andThen decodeCardHelp

decodeCardHelp constructor =
   case constructor of
      "PlayingCard" ->
         decode
            PlayingCard
               |> required "A1" decodeSuit
               |> required "A2" decodeRank
      "Joker" ->
         Dec.succeed Joker
      other->
         Dec.fail <| "Unknown constructor for type Card: " ++ other

Thanks for raising this, it simplified the code quite a bit. Let me know if anything else comes up!

[dkodaj]

The 3 issues you mentioned should new be OK (hope). You can try the copy/paste version; I’ll be back with a command-line tool soon.

[simonh1000]

Do you have any plans to make the cleanup tool work from the command line, so that we could add it to build scripts?

[dkodaj]

Yes, that would be the idea. I’m working on an npm package, should be ready in a few days.

[dkodaj]

Here’s a command-line version of the cleanup tool:
https://www.npmjs.com/package/elm-impfix

[rupert]

I don’t mean to be demanding because you’ve done a great job here. Is it possible for it to just recursively replace all .elm files under a specified folder directly? Rather than to write new files with ‘-Impfix.elm’ in the name?

Does not have to be the default behaviour and I can see why you would be careful when writing a new tool, as you don’t want to accidentally obliterate some existing .elm code that has not been checked in yet. On the other hand, we all know how to use git and revert files if something goes wrong - so I would feel quite safe running it recursively on a large project and seeing if I like the results.

[dkodaj]

Noprob about the demands, I’m happy to contribute. I’ll look into it!

[dkodaj]

I published a new patch. You can recursively overwrite all files by

$ elm-impfix "folder/**/*.elm" -r

I also fixed a few bugs that came up when I accidentally ran this command on an elm-stuff folder (not recommended).

[hopelessly]

This is a great tool. Is there any chance of a option to Decode with out needing the Json.Decode.Pipeline package and instead use the core decoders only?

[dkodaj]

Thanks!

I think that’s fairly easy to do; I’ll check. I used Pipeline because you can decode any number of fields whereas the Dec.map's run out at 8. But I guess one could make Pipeline as a last-resort option for cases when there are more than 8 fields.

[hopelessly]

Or you could use andMap http://package.elm-lang.org/packages/elm-community/json-extra/2.7.0/Json-Decode-Extra#andMap