How do you run your tests?
Just elm-test. Sorry, that’s not what I’m look for this time! But feel free to 
the next comment.
elm-test elm/test/ or elm-test src/ or elm-test "src/**/*Test.elm" or something like that. Please leave a reply below!
The reason I’m asking is because I’m working on a pull request for the elm-test runner and I would like to know more about how people are organizing their tests apart from the standard tests/ folder!
this comment if you only put tests in the tests/ folder!
I like putting test file next to the code file.
(fictional) example:
This makes it easier to find the test and also lets you know which files (modules) have tests at a glance. (I first encountered this developing angular applications, the reasoning was - that modules should be self contained - and easy to move into libraries or just copy/pasted into other applications). More info on the angular team reasoning can be found here: test file name and location
Currently the commands looks like:
elm-test src/**/ChiSquared.spec.elm
To get all files that need to be tested I’ve used glob. Some time ago I wrote a CLI tool (using typescript) to automate some angular processes - and it was general enough to work with elm.
(the basic idea was that some modules - which in my case deal with math can be easily moved with test attached)
P.S. I consider my approach a hack - but since the number of modules that I test is fairly small it generally works ok.
P.P.S. Angular applications have a config file (or more generally they use a typescript config file) to tell which files you want to test, it looks like this:
…
“include”: [
“src/**/*.spec.ts”,
“src/**/*.d.ts”
]
…
The above is the default, that is usually never changed, since the angular CLI tool automatically creates .spec.ts files. (I like the glob approach, because it allows you to organise your code in both the modular way and in the more traditional “test” folder way)
Thanks! Two things:
If you wrap the glob in quotes, elm-test will resolve the glob for you (also using glob): elm-test "src/**/ChiSquared.spec.elm"
I tried to create .spec.elm files, but I get this error:
~/stuff/test-spec
❯ npx elm-test src/ChiSquared.spec.elm
-- UNEXPECTED FILE NAME --------------------------------------------------------
I am having trouble with this file name:
/Users/lydell/stuff/test-spec/src/ChiSquared.spec.elm
I found it in your /Users/lydell/stuff/test-spec/src/ directory which is good,
but I expect all of the files in there to use the following module naming
convention:
+--------------+----------------------------------------------------+
| Module Name | File Path |
+--------------+----------------------------------------------------+
| Main | /Users/lydell/stuff/test-spec/src/Main.elm |
| HomePage | /Users/lydell/stuff/test-spec/src/HomePage.elm |
| Http.Helpers | /Users/lydell/stuff/test-spec/src/Http/Helpers.elm |
+--------------+----------------------------------------------------+
Notice that the names always start with capital letters! Can you make your file
use this naming convention?
Note: Having a strict naming convention like this makes it a lot easier to find
things in large projects. If you see a module imported, you know where to look
for the corresponding file every time!
Compilation failed while attempting to build /Users/lydell/stuff/test-spec/src/ChiSquared.spec.elm
You must be doing something more to get your setup to work?
Thank you for the first point, I did not know that glob was already used. As to the second point that was an error from my part - was looking at my angular code and added *.spec.elm automatically - in reality I’m using ChiSquaredSpec.elm
The angular style documentation rationalises the use of .spec suffix with:
I also like to think of this tests as a sort of documentation - concretely for my modules this documents how some functions behave in edge cases -> hence spec instead of test. (The ChiSquared module uses a distribution table - so it is useful to define what happens when a value is not found in the table and so on …)
Thanks! Final question area:
Which editor do you use, and do you use any Elm plugin?
If you use an Elm plugin, does it understand import Test and import Expect in your *Spec.elm files, or does it mark them as errors?
If it doesn’t mark them as errors, is it because you have elm-explorations/test in regular "dependencies" rather than in "test-dependencies" in elm.json?
We put the tests next to the tested modules in src and use elm-test 'src/**/*Test.elm'
@sebastian Thanks for the reply! Feel free to answer the editor questions as well. 
@sebastian @grigorious Would you mind trying my branch in your project? npm install --save-dev lydell/node-test-runner#js to install, then run the tests like you usually do (assuming you have a local install). I’m interested in (1) if the same amount of tests is run and (2) how long it takes*. I’m expecting a half a second speedup.
* Note that the duration that elm-test reports doesn’t include compilation time, which is what I’ve worked on. Run time npx elm-test 'src/**/*Test.elm' a couple of times before and after you’ve installed my branch to see the full duration.
I tried
elm-test
Duration: 665 ms
Passed: 880
Failed: 0
real 0m5.436s
user 0m12.894s
sys 0m7.801s
lydell/node-test-runner#js
Duration: 636 ms
Passed: 880
Failed: 0
real 0m1.496s
user 0m4.137s
sys 0m0.758s
Your fork was consistently faster. 6 secs vs 2 secs.
@Sebastian Awesome, thanks! Really nice to see that it finds all your tests and is fast. But … I now realize I forgot to say something. When I tested my branch on the test suite at work (which uses tests/) I also got several seconds better times, but then realized I wasn’t using the latest elm-test to begin with. So I updated to elm-test@0.19.1-revision4 and then speed difference shrunk to 0.5 seconds (still a win though!).
If you also have the energy to compare times for elm-test src/ (no globs, let elm-test figure out where the tests are) that would be awesome! I suspect very few people actually do that, but my PR should be as fast as elm-test@0.19.1-revision4 or a tiny bit slower (this depends on how much elm-test decides to parallelize…).
Using elm-test src/ worked, it found all the tests. It was a bit slower 2.5 secs without the glob vs 1.5secs with the glob.
Yes. I generally put tests inside the module they are meant to test.
A lot of people point out that you shouldnt test the internals of a module. If the tests are inside the module, then you have the opportunity to test internal implementation details. That seems like a valid point to me, and I keep it in mind, but I like the convenience of writing tests right next to the spot that the code is written. I really like that “side-by-side”-ness. Its also easier to make test data and test functions from within a module.
I dont know exactly off the top of my head, but I dont think I have to do anything special. My test module is usually something like this:
import Main exposing (main)
tests : Test
tests =
test "1 is 1" <| \_ -> Expect.equal 1 1
No real tests here, but it imports Main, which means that everything in Main and everything Main imports will be searched for tests. So any test in any module will be run.
This has the extra benefit of compile errors cause my tests to fail. So usually when I am developing I just run and re-run the tests to see what my compile errors.
it imports
Main, which means that everything inMainand everythingMainimports will be searched for tests. So any test in any module will be run.
No, that’s not how elm-test works. It only looks for exposed Test values in the CLI args (which defaults to tests/ – a directory arg means “all .elm files deeply within this folder”). I just tried it just to be 100% sure.
So you must be running something like elm-test tests/ src/, or you’re not running as many tests as you think you are. Or maybe you do something like this?
module SomeTest exposing (..)
import Expect
import Main
import Test exposing (..)
tests : Test
tests =
describe "SomeTest"
[ Main.tests
, test "something else" whatever
]
I am positive that I have this working, because I run tests like this and they do in fact break when things go wrong.
So, I do still have to have a tests/MainTests.elm file. But that file doesnt do anything; it looks just like what I posted before. MainTests.elm merely has import Main at the top. I just read through the tests we have at work, and I notice modules expose their tests tests : Test, so I think maybe that is key.
But, they arent explicitly organized into a heirarchy like with describe "Tests" [ Main.tests ]. I think elm-test really does climb the module heirarchy and run any exposed Test values.
@Chadtech I can’t get that to work: https://github.com/lydell/elm-test-import
Also I see nothing in the elm-test code that suggests it should work.
Which version of elm-test/node-test-runner do you use?
Update:
It looks like we’re both right.
elm-test 0.19.1 and 0.19.1-revision2 both work like @Chadtech says.
elm-test 0.19.1-revision3 and 0.19.1-revision4 work like I say.
Sounds like behavior accidentally changed in 0.19.1-revision3. @Chadtech Can you confirm that you’re using elm-test 0.19.1 or 0.19.1-revision2?