Running tests for elm/core

[jerith]

I’m hoping to work on a fix for this elm/core bug.

Seems like the right place to start is to run the existing test suite. Running elm-test produced some errors, though.

The first couple were easy to fix; I made these changes on top of the current tip of its master branch (22b2d7a2):

diff --git a/elm.json b/elm.json
index 745d790..1d9f9cc 100644
--- a/elm.json
+++ b/elm.json
@@ -35,5 +35,7 @@
     },
     "elm-version": "0.19.0 <= v < 0.20.0",
     "dependencies": {},
-    "test-dependencies": {}
-}
\ No newline at end of file
+    "test-dependencies": {
+        "elm-explorations/test": "1.2.1 <= v < 2.0.0"
+    }
+}
diff --git a/tests/Test/List.elm b/tests/Test/List.elm
index ed26f0f..d5fd9d2 100644
--- a/tests/Test/List.elm
+++ b/tests/Test/List.elm
@@ -3,7 +3,7 @@ module Test.List exposing (tests)
 import Test exposing (..)
 import Expect
 import Basics exposing (..)
-import Maybe exposing (Maybe(Nothing, Just))
+import Maybe exposing (Maybe(..))
 import List exposing (..)

That gets me to this error, which has me stumped:

$ rm -rf .elm && ELM_HOME=$(pwd)/.elm ./node_modules/elm-test/bin/elm-test 
-- AMBIGUOUS IMPORT ------------- /home/matt/git/elm/core/tests/Test/Bitwise.elm

The Test.Bitwise module has an ambiguous import:

    import Bitwise

I found multiple module with that name though:

    /home/matt/git/elm/core/src/Bitwise.elm
    exposed by elm/core 1.0.0

Which is the right one? Try renaming your modules to have unique names.

Compilation failed while attempting to build /home/matt/git/elm/core/tests/Main.elm /home/matt/git/elm/core/tests/Test/Array.elm /home/matt/git/elm/core/tests/Test/Basics.elm /home/matt/git/elm/core/tests/Test/Bitwise.elm /home/matt/git/elm/core/tests/Test/Char.elm /home/matt/git/elm/core/tests/Test/CodeGen.elm /home/matt/git/elm/core/tests/Test/Dict.elm /home/matt/git/elm/core/tests/Test/Equality.elm /home/matt/git/elm/core/tests/Test/List.elm /home/matt/git/elm/core/tests/Test/Maybe.elm /home/matt/git/elm/core/tests/Test/Result.elm /home/matt/git/elm/core/tests/Test/Set.elm /home/matt/git/elm/core/tests/Test/String.elm /home/matt/git/elm/core/tests/Test/Tuple.elm

There is a tests/run-tests.sh script, but clearly it hasn’t been updated for 0.19 yet (it still references elm-make etc. and git history says it was last touched in 2016).

That run-tests.sh script seems like it’s trying to fake out the elm/core directory inside the old elm-stuff/ directory. So I tried something similar based on my understanding of how the analogous mechanism works in 0.19:

$ rm -rf .elm/0.19.0/package/elm/core/1.0.0 && ln -s $(pwd) .elm/0.19.0/package/elm/core/1.0.0

$ ELM_HOME=$(pwd)/.elm ./node_modules/elm-test/bin/elm-test 
elm: getCurrentDirectory:getWorkingDirectory: does not exist (Current working directory no longer exists)
elm: getCurrentDirectory:getWorkingDirectory: does not exist (Current working directory no longer exists)
elm: getCurrentDirectory:getWorkingDirectory: does not exist (Current working directory no longer exists)
elm: getCurrentDirectory:getWorkingDirectory: does not exist (Current working directory no longer exists)
elm: thread blocked indefinitely in an MVar operation
Compilation failed while attempting to build /home/matt/git/elm/core/tests/Main.elm /home/matt/git/elm/core/tests/Test/Array.elm /home/matt/git/elm/core/tests/Test/Basics.elm /home/matt/git/elm/core/tests/Test/Bitwise.elm /home/matt/git/elm/core/tests/Test/Char.elm /home/matt/git/elm/core/tests/Test/CodeGen.elm /home/matt/git/elm/core/tests/Test/Dict.elm /home/matt/git/elm/core/tests/Test/Equality.elm /home/matt/git/elm/core/tests/Test/List.elm /home/matt/git/elm/core/tests/Test/Maybe.elm /home/matt/git/elm/core/tests/Test/Result.elm /home/matt/git/elm/core/tests/Test/Set.elm /home/matt/git/elm/core/tests/Test/String.elm /home/matt/git/elm/core/tests/Test/Tuple.elm

So, is there a way to run these tests?

[jerith]

I’ve gotten a tiny bit farther, by basically creating an entirely separate (empty) project with these tests. So they’ll be testing the elm/core that comes from package.elm-lang.org not from the local project.

(I think I can later get around that by running the tests under nix where I can insert local copies of the elm/core source into $ELM_HOME.)

Anyway, doing this indicates that these tests have clearly not been updated from 0.18 to 0.19 syntax. I think I’m going to start on that work, but it’s obviously kind of a big task, so I don’t want to repeat work if it’s already been done elsewhere. Anybody know if has?

[mfeineis]

The last time I heard something about tests for the core stuff - including the compiler - was that they were not found useful enough to have them. I personally don’t agree with that notion, good tests are never a waste of time, especially regression tests. I too would be very interested in case anybody has more up-to-date information on that topic.

[jerith]

I’ve gotten the tests running over in the update-019 branch in my fork of elm-core. However they’re testing the code from package.elm-lang.org, not the code in the local git working copy.

This did reveal an interesting change from 0.18 to 0.19 that AFAIK isn’t documented anywhere: hex strings (e.g. “0xBEEF”) no longer parse with String.toInt.

I’ve also been able to build upon this to create a test for the bug I’m interested in. So I’ll keep working on making these 0.19-compatible tests actually run against the local git working copy, so I can develop and verify a fix.

[jerith]

update run-tests.sh to work with 0.19