Community BETA for Elm 0.19.1

[evancz]

Keep discussion on Discourse and Slack. Please do not post on HN, reddit, twitter, etc.

I am hoping to get some feedback on elm-0.19.1-beta-1

The goal of 0.19.1 is generally to clean up the rough edges introduced in 0.19.0 such that we have a really solid foundation for newcomers, professionals, scientists, etc.

The goal of the BETA is to get some of the more engaged community members to try it out and find any problems before a real release that’ll go out to a broader audience. I do not recommend switching your company to using the BETA. Wait for the real release! Only try out the BETA if you want to help test it out.

Improvements

Notable improvements include:

• Parse error message quality (like this and this)
• Faster compilation, especially for incremental compiles
• Uses filelocks so that cached files are not corrupted when plugins run elm make multiple times on the same project at the same time. (Still worth avoiding that though!)
• More intuitive multiline declarations in REPL
• Various bug fixes

There are no language changes, so once you swap the "elm-version" in your elm.json, most users should be able to proceed without any further code changes. You may run into a handful of bugfixes though! I will outline them in the next comment in this thread!

Binaries

Say you want to try elm-0.19.1-beta-1 on your ~/Documents/hats/ project. Download the binary from the following links to ~/Documents/hats/elm and run ./elm make to try it out. No need to change your PATH.

• Mac
• Linux
• Windows

While I do highly appreciate people trying these out and helping with the goals listed below, I do not recommend using these in production! They have specific limitations (e.g. elm publish is disabled for now) and may have problems that we have not figured out yet.

Goals

In this testing period, I am hoping to get help on the following topics:

1. Check bug fixes. A couple things started going wrong in 0.19.0, like --debug, x /= 0, type Height = Height Float in --optimize, etc. If you experienced any of these, please check that it is resolved with the new binary and report back!

2. Time compiler with 50k+ LOC projects. The new binaries should be a decent bit faster and should take advantage of multiple cores better. I am hoping to hear some before-and-after numbers just to confirm that this is the case.

3. Cause parse errors. Please try to cause parse errors and let us know if you find something interesting. Maybe there is a message that seems off or not helpful enough. That kind of thing.

4. Try out the REPL. It should be more intuitive to make multi-line expressions. Try it out and report back how it feels.

To report anything that may need changes, please create an SSCCE and report your findings in this thread on discourse or in the #core-coordination channel on slack.

---

Note: There is some supporting tooling that is not compatible yet. This is okay! The point of this time is so that people can get prepared without a bunch of people telling them to hurry up. For example, a bit more work is needed on elm-test, and I’m sure there are editors with fancy features that are not updated yet. This is all as expected! This is part of why I do not recommend switching to the BETA at work.

---

Thank you to anyone who gives the BETA a try, and please keep discussion on discourse and slack. There may still be problems, so I’d like to do more testing before this reaches a broader audience through HN, reddit, twitter, etc. I’ll make a post on elm-lang.org and share it on twitter when it is ready for a broader release like that!

And thank you to the folks who helped test the ALPHA. Very glad to find those issues early!

[evancz]

Detectable Bug Fixes

A couple of the fixes in 0.19.1 may be detectable in code that compiles with 0.19.0. To say it another way, there are three known cases where code that compiled with 0.19.0 will not compile with the BETA binaries:

1. Ambiguous Imports

Say you have an import like this:

import Html exposing (min)
import Regex exposing (never)

x = min
y = never

These should be reported as ambiguous usages since the names are also exposed by Basics, but there was a regression in 0.19.0 described here such that they weren’t caught in specific circumstances

The fix is to use a qualified name like Html.min or Regex.never to make it unambiguous.

We found a couple instances of this in packages and have submitted PRs to the relevant authors to fix it before 0.19.1 is released for real. You may run into this in your own code as well.

For more details on why this is considered a regression, check out the details here or try it in 0.18.0 to see how it worked before.

2. Tabs in Comments

The 0.19.0 binaries did not catch tab characters in comments. The new parser is better at checking for tabs, so it will object when it finds these.

Again, we found this in some packages and have already reached out to the relevant authors with PRs so we can get a patch published before 0.19.1 is released publicly.

3. Port Module with no Ports

If you have any files that start with:

port module Main exposing (..)

But they do not actually have any port declarations, the 0.19.1 binary will ask you to switch to a normal module declaration like module Main exposing (..)

---

That is everything that is known based on the ALPHA testing. Please tell us about it if you find any other cases that seem curious.

[Libbum]

Upon quick, initial testing I can confirm that the Map! bug in --debug mode is gone from all the six projects of mine that were afflicted with that.

Will play around with it further over the next few days and try to identify any rough edges.

Great work!

[Jess_Bromley]

In 0.19 I often got an error message of the form “this call produces a x, but the type annotation says it should be a y” where x and y were in fact identical!

I eventually worked out that the simplest way this was happening was when I accidentally provided a function instead of the result of that function as an argument to something, when the answer itself still needed to be a function. Probably an example explains this better. Under 0.19, the following:

makeTranslator : String -> String -> String
makeTranslator =
    Debug.todo ""


translator : String -> String
translator =
    makeTranslator

produces the error:

Something is off with the body of the translator definition:

11| makeTranslator
^^^^^^^^^^^^^^
This makeTranslator value is a:

String → String

But the type annotation on translator says it should be:

String → String

I can confirm that under 0.19.1 I now get a correct error message, namely:

Something is off with the body of the translator definition:

11| makeTranslator
^^^^^^^^^^^^^^
This makeTranslator value is a:

String → String → String

But the type annotation on translator says it should be:

String → String

Hint: It looks like it takes too many arguments. I see 1 extra.

However, when this error is triggered in more complicated code, the provided hint isn’t always appropriate. Consider the following example, which is a reduction of a real world error I encountered when trying to understand Url.Parser…

type Parser x y
    = Parser x y


string : Parser (String -> a) a
string =
    Debug.todo ""


slash : Parser a b -> Parser b c -> Parser a c
slash =
    Debug.todo ""


test : Parser (String -> String) String
test =
    slash string string

Under 0.19 this again produced an incorrect error message of:

Something is off with the body of the test definition:

20| slash string string
^^^^^^^^^^^^^^^^^^^
This slash call produces:

Parser (String → String) String

But the type annotation on test says it should be:

Parser (String → String) String

Under 0.19.1 the error message (below) is improved again, however the hint is (in my opinion) now very confusing for someone trying to understand what is going wrong (what takes too many arguments?)…

Something is off with the body of the test definition:

20| slash string string
^^^^^^^^^^^^^^^^^^^
This slash call produces:

Parser (String → String → String) String

But the type annotation on test says it should be:

Parser (String → String) String

Hint: It looks like it takes too many arguments. I see 1 extra.

But that’s just my opinion, possibly others don’t think it so confusing. The error message is still vastly improved over 0.19!

[evancz]

@Jess_Bromley, as I was going through issues recently, I found some reports of the issue you mention about functions with different numbers of arguments showing up wrong in type error messages. So that one I was able to fix!

There is some logic to detect specific differences and give hints based on that. I wanted to improve that logic for some other scenario, but I found that it’d end up being quite a significant change. Significant changes are really risky with types, so I think it’s best to make changes like that earlier in a release cycle.

So for now, would you mind reporting the Parser error message to https://github.com/elm/error-message-catalog/issues? That has been a really helpful way to figure out where the problems are, and recording it there has a great track record of inspiring big-picture improvements!

[jfairbank]

Awesome! Thank you, Evan, for all your hard work to fix compiler issues and improve error messaging.

I regret not testing with the alpha so I could bring this up earlier. I really appreciate the change to better support multiline expressions in the REPL, but now the old multiline syntax with \ doesn’t seem to work. It interprets it as the start to a lambda expression. I’m sure it’s open to interpretation since this is a change to the REPL and not the actual language, but that seems to me like a breaking change.

Would you consider (are you able) to support the old multiline syntax until the next minor/major version of Elm? Being fully transparent, this mainly gives me concern because Programming Elm uses the old \ syntax, so readers will run into syntax errors the very first chapter as well as two other chapters later in the book. I’m not sure when I will be able to release a book update with Pragmatic, and it’s unfortunate that those currently with copies of the book won’t be able to run early examples in the book if they don’t know to go to the book’s errata webpage. If you support the old syntax, then those readers won’t encounter issues, and I can still update the code samples to use the new multiline format for future readers.

Thank you for considering this if it’s possible!

[rupert]

@evancz

Time compiler with 50k+ LOC projects.

On a 524k LOC project:

for 0.19.1-beta-1:

> time elm-0.19.1-beta-1-linux make

> real    0m4.327s

and for 0.19:

> time elm make

> real    0m12.745s

In case you are wondering if I have a 524k LOC project in production - no I don’t. This is generated code that produces all the encoders/decoders and API stubs for AWS; I am currently working on upgrading ktonon/elm-aws-generate to 0.19; lots of services, lots of code.

12 seconds was already impressive, 4 is delightfully refined.

Thanks Evan and core team for all your hard work and sticking to your principals. 0.19 has been a very productive experience for me, and it is pleasing to see it mature into a 0.19.1 release.

[Namek]

How about rough edges around updating dependencies? Updating dependencies is not straightforward

It’s very unfortunate that for such a simple task we need external tools and/or removing cache. I once told about this to my colleague and he had impression of very immature technology. Can’t convince people that compiler is awesome if things around are not even OK.

[rupert]

@Namek

How about rough edges around updating dependencies? Updating dependencies is not straightforward

Would definitely be welcome if there is going to be a 0.19.2 release…

[evancz]

@rupert, very interesting, thank you for running these numbers!

@jfairbank, I’ll reach out to discuss with you.

@Namek, I have done some prototyping on how to improve install/uninstall, but unfortunately it is not possible to improve everything that needs improving in one release. I think the problem you mention is well understood, but if you feel it needs further elaboration, please start a new thread.

[jxxcarlson]

I’ve been using Ilias’ elm-json:

• elm-json install XXX
• elm-json uninstall XXX
• elm-json upgrade

and have been very happy with it.

[hecrj]

Here are my results on a 45k+ LOC codebase, using a Ryzen 9 3900X:

0.19

$ time elm make src/Main.elm
Dependencies loaded from local cache.
Dependencies ready!                
Success! Compiled 228 modules.                                       
Done in 1.97s.

real    0m2.116s
user    0m1.433s
sys     0m0.068s

0.19.1-beta-1

$ time ./elm-0.19.1-beta-1-linux make src/Main.elm 
Dependencies ready!           
Success! Compiled 228 modules.

    Main ───> index.html


real    0m1.152s
user    0m0.915s
sys     0m0.818s

When 0.19 was released, I thought it couldn’t get any faster… Guess I was wrong!

[lucamug]

On a 28k LOC (sorry, less than 50k)

0.19.0

128 modules 3.385 secs
  1 module  0.740 secs

0.19.1

128 modules 1.941 secs
  1 module  0.314 secs

       Full compile: 43% decrease!
Incremental compile: 58% decrease!

[lenards]

I can confirm on Windows 10 that the Map! bug when using --debug flag is resolved the project I work on.

Thank you so much for this beta. So happy to have the debugger back.

[Birowsky]

Holly effin durians! I just migrated from 0.18.0 on a 51K app where changing this one critical module caused ~3min compile time. And now with 0.19.0:

Success! Compiled 197 modules.                                       

real    0m5.324s
user    0m4.845s
sys     0m0.436s

But then there is 0.19.1:

Success! Compiled 1 module.

real    0m0.158s
user    0m0.060s
sys     0m0.082s

Are. You. Kidding. Me.
What have you done, Evan!

[jerith]

I’m pretty sure I won’t have time to test this particular beta, but in the future, it’d be easier for those of us on source-based distros (like NixOS) to test betas if you publish a tag in the git repo. Thanks!

[sashaafm]

Aren’t you comparing the compilation of 197 modules vs. 1 module? Most likely increment compilation? Or am I misunderstanding something?

[Birowsky]

Incremental is the only type that matters to me. I believe I made sure I edited just that one key module before running the compiler. That accurately describes my worst case scenario.

[FranzSkuffka]

To test it with create-elm-app:

1. Inject new elm runtime

~/n/lib/node_modules/create-elm-app/node_modules/elm/unpacked_bin   
❯ mv elm elm_019
                                                         
~/n/lib/node_modules/create-elm-app/node_modules/elm/unpacked_bin   
❯ cp ~/Downloads/elm-0.19.1-beta-1-mac elm

                                                         
~/n/lib/node_modules/create-elm-app/node_modules/elm/unpacked_bin   
❯ sudo chmod +x elm

2. Adjust hot loader
   ❯ vi elm-hot/src/inject.js # on line 35

    if (false) {
        throw new Error("Compiled JS from the Elm compiler is not valid. You must use the Elm 0.19 compiler.");
    }

3. Change the elm version in the elm-app created project folder to 0.19.1

[evancz]

@sashaafm

Aren’t you comparing the compilation of 197 modules vs. 1 module? Most likely increment compilation? Or am I misunderstanding something?

Those are both for incremental compilation times. One of the big improvements in 0.19.1 is that it checks if the public API of a module has changed. If it has not changed, there is no need to recompile any of the modules that import it.

So @Birowsky is seeing it go from 197 modules to 1 module as a result of this change. There are still 196 modules that depend on the API of this one critical module, but the compiler can skip them for most edits!

---

Note 1: You will be able to increase the benefits of this optimization by using an explicit exposing in your modules. That way you can add new helper functions internally without changing the public API, thereby getting the faster compiles in more cases.

---

Note 2: This optimization also means that the worst case is improved. When an public API is changed, you must check modules that import it directly. Maybe they are broken. But you do not need to check on any modules that import it indirectly. So if your project was like:

   BigModule
    /     \
   A       B
  / \     / \
 C   D   E   F 
/ \ / \ / \ / \
................

When you change the public API of BigModule you only need to check on A and B. Their API stays the same, so you can skip compiling C, D, E, F, and everything else. So even in the worst case, it is unlikely that @Birowsky could trigger a 197 module recompile when using the new compiler.