# Do you put tests outside the \`tests/\` folder?

**URL:** <https://discourse.elm-lang.org/t/do-you-put-tests-outside-the-tests-folder/6386>\
**Category:** Request Feedback\
**Created:** [October 9, 2020, 3:37pm UTC](https://discourse.elm-lang.org/t/do-you-put-tests-outside-the-tests-folder/6386 "2020-10-09T15:37:17Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![lydell](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.elm-lang.org/lydell/32/178_2.png) [@lydell](https://discourse.elm-lang.org/u/lydell)\
**Post date:** [October 9, 2020, 3:37pm UTC](https://discourse.elm-lang.org/t/do-you-put-tests-outside-the-tests-folder/6386/1 "2020-10-09T15:37:17Z")

</div>

How do you run your tests?

1. Just `elm-test`. Sorry, that’s not what I’m look for this time! But feel free to ❤ the next comment.

2. `elm-test elm/test/` or `elm-test src/` or `elm-test "src/**/*Test.elm"` or something like that. Please leave a reply below!

The reason I’m asking is because I’m working on a pull request for the elm-test runner and I would like to know more about how people are organizing their tests apart from the standard `tests/` folder!

> <https://github.com/rtfeldman/node-test-runner/pull/442>

---

<div class="post-metadata">

**Author:** ![lydell](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.elm-lang.org/lydell/32/178_2.png) [@lydell](https://discourse.elm-lang.org/u/lydell)\
**Post date:** [October 9, 2020, 3:37pm UTC](https://discourse.elm-lang.org/t/do-you-put-tests-outside-the-tests-folder/6386/2 "2020-10-09T15:37:38Z")

</div>

❤ this comment if you only put tests in the `tests/` folder!

---

<div class="post-metadata">

**Author:** ![grigorious](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.elm-lang.org/grigorious/32/3549_2.png) [@grigorious](https://discourse.elm-lang.org/u/grigorious)\
**Post date:** [October 11, 2020, 11:26am UTC](https://discourse.elm-lang.org/t/do-you-put-tests-outside-the-tests-folder/6386/3 "2020-10-11T11:26:14Z")

</div>

I like putting test file next to the code file.  
(fictional) example:

- ChiSquared.elm // the module
- ChiSquared.spec.elm // the specifications (unit tests) for the module

This makes it easier to find the test and also lets you know which files (modules) have tests at a glance. (I first encountered this developing angular applications, the reasoning was - that modules should be self contained - and easy to move into libraries or just copy/pasted into other applications). More info on the angular team reasoning can be found here: [test file name and location](https://angular.io/guide/testing#test-file-name-and-location)

---

<div class="post-metadata">

**Author:** ![lydell](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.elm-lang.org/lydell/32/178_2.png) [@lydell](https://discourse.elm-lang.org/u/lydell)\
**Post date:** [October 11, 2020, 11:54am UTC](https://discourse.elm-lang.org/t/do-you-put-tests-outside-the-tests-folder/6386/4 "2020-10-11T11:54:12Z")

</div>

@grigorious Cool! Can you share the CLI command you use to run the tests?

---

<div class="post-metadata">

**Author:** ![grigorious](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.elm-lang.org/grigorious/32/3549_2.png) [@grigorious](https://discourse.elm-lang.org/u/grigorious)\
**Post date:** [October 11, 2020, 2:19pm UTC](https://discourse.elm-lang.org/t/do-you-put-tests-outside-the-tests-folder/6386/5 "2020-10-11T14:19:07Z")

</div>

Currently the commands looks like:

`elm-test src/**/ChiSquared.spec.elm`

To get all files that need to be tested I’ve used [glob](https://github.com/isaacs/node-glob#readme). Some time ago I wrote a CLI tool (using typescript) to automate some angular processes - and it was general enough to work with elm.

(the basic idea was that some modules - which in my case deal with math can be easily moved with test attached)

P.S. I consider my approach a hack - but since the number of modules that I test is fairly small it generally works ok.

P.P.S. Angular applications have a config file (or more generally they use a typescript config file) to tell which files you want to test, it looks like this:

…  
“include”: [  
“src/\*\*/\*.spec.ts”,  
“src/\*\*/\*.d.ts”  
]  
…

The above is the default, that is usually never changed, since the angular CLI tool automatically creates `.spec.ts` files. (I like the glob approach, because it allows you to organise your code in both the modular way and in the more traditional “test” folder way)

---

<div class="post-metadata">

**Author:** ![lydell](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.elm-lang.org/lydell/32/178_2.png) [@lydell](https://discourse.elm-lang.org/u/lydell)\
**Post date:** [October 11, 2020, 4:07pm UTC](https://discourse.elm-lang.org/t/do-you-put-tests-outside-the-tests-folder/6386/6 "2020-10-11T16:07:42Z")

</div>

Thanks! Two things:

1. If you wrap the glob in quotes, elm-test will resolve the glob for you (also using [glob](https://github.com/isaacs/node-glob#readme)): `elm-test "src/**/ChiSquared.spec.elm"`

2. I tried to create .spec.elm files, but I get this error:

```auto
~/stuff/test-spec
❯ npx elm-test src/ChiSquared.spec.elm
-- UNEXPECTED FILE NAME --------------------------------------------------------

I am having trouble with this file name:

    /Users/lydell/stuff/test-spec/src/ChiSquared.spec.elm

I found it in your /Users/lydell/stuff/test-spec/src/ directory which is good,
but I expect all of the files in there to use the following module naming
convention:

    +--------------+----------------------------------------------------+
    | Module Name | File Path |
    +--------------+----------------------------------------------------+
    | Main | /Users/lydell/stuff/test-spec/src/Main.elm |
    | HomePage | /Users/lydell/stuff/test-spec/src/HomePage.elm |
    | Http.Helpers | /Users/lydell/stuff/test-spec/src/Http/Helpers.elm |
    +--------------+----------------------------------------------------+

Notice that the names always start with capital letters! Can you make your file
use this naming convention?

Note: Having a strict naming convention like this makes it a lot easier to find
things in large projects. If you see a module imported, you know where to look
for the corresponding file every time!

Compilation failed while attempting to build /Users/lydell/stuff/test-spec/src/ChiSquared.spec.elm

```

You must be doing something more to get your setup to work?

---

<div class="post-metadata">

**Author:** ![grigorious](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.elm-lang.org/grigorious/32/3549_2.png) [@grigorious](https://discourse.elm-lang.org/u/grigorious)\
**Post date:** [October 11, 2020, 7:14pm UTC](https://discourse.elm-lang.org/t/do-you-put-tests-outside-the-tests-folder/6386/7 "2020-10-11T19:14:00Z")

</div>

Thank you for the first point, I did not know that glob was already used. As to the second point that was an error from my part - was looking at my angular code and added `*.spec.elm` automatically - in reality I’m using `ChiSquaredSpec.elm`

The angular style [documentation](https://angular.io/guide/styleguide#unit-test-file-names) rationalises the use of .spec suffix with:

- Provides a consistent way to quickly identify tests.
- Provides pattern matching for test runners

I also like to think of this tests as a sort of documentation - concretely for my modules this documents how some functions behave in edge cases -\> hence **spec** instead of test. (The ChiSquared module uses a distribution table - so it is useful to define what happens when a value is not found in the table and so on …)

---

<div class="post-metadata">

**Author:** ![lydell](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.elm-lang.org/lydell/32/178_2.png) [@lydell](https://discourse.elm-lang.org/u/lydell)\
**Post date:** [October 11, 2020, 7:52pm UTC](https://discourse.elm-lang.org/t/do-you-put-tests-outside-the-tests-folder/6386/8 "2020-10-11T19:52:33Z")

</div>

Thanks! Final question area:

1. Which editor do you use, and do you use any Elm plugin?

2. If you use an Elm plugin, does it understand `import Test` and `import Expect` in your `*Spec.elm` files, or does it mark them as errors?

3. If it doesn’t mark them as errors, is it because you have `elm-explorations/test` in regular `"dependencies"` rather than in `"test-dependencies"` in elm.json?

---

<div class="post-metadata">

**Author:** ![Sebastian](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.elm-lang.org/sebastian/32/98_2.png) [@Sebastian](https://discourse.elm-lang.org/u/Sebastian)\
**Post date:** [October 11, 2020, 11:18pm UTC](https://discourse.elm-lang.org/t/do-you-put-tests-outside-the-tests-folder/6386/9 "2020-10-11T23:18:10Z")

</div>

We put the tests next to the tested modules in `src` and use `elm-test 'src/**/*Test.elm'`

---

<div class="post-metadata">

**Author:** ![lydell](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.elm-lang.org/lydell/32/178_2.png) [@lydell](https://discourse.elm-lang.org/u/lydell)\
**Post date:** [October 12, 2020, 6:13am UTC](https://discourse.elm-lang.org/t/do-you-put-tests-outside-the-tests-folder/6386/10 "2020-10-12T06:13:19Z")

</div>

@sebastian Thanks for the reply! Feel free to answer the editor questions as well. 🙏

@sebastian @grigorious Would you mind trying my branch in your project? `npm install --save-dev lydell/node-test-runner#js` to install, then run the tests like you usually do (assuming you have a local install). I’m interested in (1) if the same amount of tests is run and (2) how long it takes\*. I’m expecting a half a second speedup.

\* Note that the duration that elm-test reports doesn’t include compilation time, which is what I’ve worked on. Run `time npx elm-test 'src/**/*Test.elm'` a couple of times before and after you’ve installed my branch to see the full duration.

---

<div class="post-metadata">

**Author:** ![Sebastian](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.elm-lang.org/sebastian/32/98_2.png) [@Sebastian](https://discourse.elm-lang.org/u/Sebastian)\
**Post date:** [October 12, 2020, 10:39pm UTC](https://discourse.elm-lang.org/t/do-you-put-tests-outside-the-tests-folder/6386/11 "2020-10-12T22:39:35Z")

</div>

I tried

**elm-test**

Duration: 665 ms  
Passed: 880  
Failed: 0

real 0m5.436s  
user 0m12.894s  
sys 0m7.801s

**lydell/node-test-runner#js**

Duration: 636 ms  
Passed: 880  
Failed: 0

real 0m1.496s  
user 0m4.137s  
sys 0m0.758s

Your fork was consistently faster. 6 secs vs 2 secs.

---

<div class="post-metadata">

**Author:** ![lydell](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.elm-lang.org/lydell/32/178_2.png) [@lydell](https://discourse.elm-lang.org/u/lydell)\
**Post date:** [October 13, 2020, 6:46am UTC](https://discourse.elm-lang.org/t/do-you-put-tests-outside-the-tests-folder/6386/12 "2020-10-13T06:46:32Z")

</div>

@Sebastian Awesome, thanks! Really nice to see that it finds all your tests and is fast. But … I now realize I forgot to say something. When I tested my branch on the test suite at work (which uses `tests/`) I also got several seconds better times, but then realized I wasn’t using the latest elm-test to begin with. So I updated to `elm-test@0.19.1-revision4` and then speed difference shrunk to 0.5 seconds (still a win though!).

If you also have the energy to compare times for `elm-test src/` (no globs, let elm-test figure out where the tests are) that would be awesome! I suspect very few people actually do that, but my PR should be as fast as `elm-test@0.19.1-revision4` or a tiny bit slower (this depends on how much elm-test decides to parallelize…).

---

<div class="post-metadata">

**Author:** ![Sebastian](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.elm-lang.org/sebastian/32/98_2.png) [@Sebastian](https://discourse.elm-lang.org/u/Sebastian)\
**Post date:** [October 13, 2020, 11:50pm UTC](https://discourse.elm-lang.org/t/do-you-put-tests-outside-the-tests-folder/6386/13 "2020-10-13T23:50:03Z")

</div>

Using `elm-test src/` worked, it found all the tests. It was a bit slower 2.5 secs without the glob vs 1.5secs with the glob.

---

<div class="post-metadata">

**Author:** ![Chadtech](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.elm-lang.org/chadtech/32/609_2.png) [@Chadtech](https://discourse.elm-lang.org/u/Chadtech)\
**Post date:** [October 19, 2020, 12:09am UTC](https://discourse.elm-lang.org/t/do-you-put-tests-outside-the-tests-folder/6386/14 "2020-10-19T00:09:24Z")

</div>

Yes. I generally put tests inside the module they are meant to test.

A lot of people point out that you shouldnt test the internals of a module. If the tests are inside the module, then you have the opportunity to test internal implementation details. That seems like a valid point to me, and I keep it in mind, but I like the convenience of writing tests right next to the spot that the code is written. I really like that “side-by-side”-ness. Its also easier to make test data and test functions from within a module.

---

<div class="post-metadata">

**Author:** ![lydell](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.elm-lang.org/lydell/32/178_2.png) [@lydell](https://discourse.elm-lang.org/u/lydell)\
**Post date:** [October 19, 2020, 5:24am UTC](https://discourse.elm-lang.org/t/do-you-put-tests-outside-the-tests-folder/6386/15 "2020-10-19T05:24:27Z")

</div>

@Chadtech Thanks for your answer! What does your `elm-test` command look like?

---

<div class="post-metadata">

**Author:** ![Chadtech](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.elm-lang.org/chadtech/32/609_2.png) [@Chadtech](https://discourse.elm-lang.org/u/Chadtech)\
**Post date:** [October 19, 2020, 6:17am UTC](https://discourse.elm-lang.org/t/do-you-put-tests-outside-the-tests-folder/6386/16 "2020-10-19T06:17:46Z")

</div>

I dont know exactly off the top of my head, but I dont think I have to do anything special. My test module is usually something like this:

```elm
import Main exposing (main)

tests : Test
tests =
    test "1 is 1" <| \_ -> Expect.equal 1 1

```

No real tests here, but it imports `Main`, which means that everything in `Main` and everything `Main` imports will be searched for tests. So any test in any module will be run.

This has the extra benefit of compile errors cause my tests to fail. So usually when I am developing I just run and re-run the tests to see what my compile errors.

---

<div class="post-metadata">

**Author:** ![lydell](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.elm-lang.org/lydell/32/178_2.png) [@lydell](https://discourse.elm-lang.org/u/lydell)\
**Post date:** [October 19, 2020, 8:27pm UTC](https://discourse.elm-lang.org/t/do-you-put-tests-outside-the-tests-folder/6386/17 "2020-10-19T20:27:10Z")

</div>

> it imports `Main` , which means that everything in `Main` and everything `Main` imports will be searched for tests. So any test in any module will be run.

No, that’s not how elm-test works. It only looks for exposed `Test` values in the CLI args (which defaults to `tests/` – a directory arg means “all .elm files deeply within this folder”). I just tried it just to be 100% sure.

So you must be running something like `elm-test tests/ src/`, or you’re not running as many tests as you think you are. Or maybe you do something like this?

```elm
module SomeTest exposing (..)

import Expect
import Main
import Test exposing (..)

tests : Test
tests =
    describe "SomeTest"
        [ Main.tests
        , test "something else" whatever
        ]

```

---

<div class="post-metadata">

**Author:** ![Chadtech](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.elm-lang.org/chadtech/32/609_2.png) [@Chadtech](https://discourse.elm-lang.org/u/Chadtech)\
**Post date:** [October 19, 2020, 9:13pm UTC](https://discourse.elm-lang.org/t/do-you-put-tests-outside-the-tests-folder/6386/18 "2020-10-19T21:13:23Z")

</div>

I am positive that I have this working, because I run tests like this and they do in fact break when things go wrong.

So, I do still have to have a `tests/MainTests.elm` file. But that file doesnt do anything; it looks just like what I posted before. `MainTests.elm` merely has `import Main` at the top. I just read through the tests we have at work, and I notice modules expose their tests `tests : Test`, so I think maybe that is key.

But, they arent explicitly organized into a heirarchy like with `describe "Tests" [Main.tests]`. I think `elm-test` really does climb the module heirarchy and run any exposed `Test` values.

---

<div class="post-metadata">

**Author:** ![lydell](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.elm-lang.org/lydell/32/178_2.png) [@lydell](https://discourse.elm-lang.org/u/lydell)\
**Post date:** [October 20, 2020, 7:21am UTC](https://discourse.elm-lang.org/t/do-you-put-tests-outside-the-tests-folder/6386/19 "2020-10-20T07:21:55Z")

</div>

@Chadtech I can’t get that to work: [https://github.com/lydell/elm-test-import](https://github.com/lydell/elm-test-import)

Also I see nothing in the elm-test code that suggests it should work.

Which version of elm-test/node-test-runner do you use?

---

<div class="post-metadata">

**Author:** ![lydell](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.elm-lang.org/lydell/32/178_2.png) [@lydell](https://discourse.elm-lang.org/u/lydell)\
**Post date:** [October 20, 2020, 4:10pm UTC](https://discourse.elm-lang.org/t/do-you-put-tests-outside-the-tests-folder/6386/20 "2020-10-20T16:10:27Z")

</div>

Update:

It looks like we’re both right.

elm-test 0.19.1 and 0.19.1-revision2 both work like @Chadtech says.

elm-test 0.19.1-revision3 and 0.19.1-revision4 work like I say.

Sounds like behavior accidentally changed in 0.19.1-revision3. @Chadtech Can you confirm that you’re using elm-test 0.19.1 or 0.19.1-revision2?

[Next page](https://discourse.elm-lang.org/t/do-you-put-tests-outside-the-tests-folder/6386.md?page=2)