Elmer: An Elm Testing Tool

I’m not sure we agree. When I’m using Elmer to test-drive an application, I’m not testing real commands and subscriptions directly at all. Instead, my tests describe how my code interacts with the functions that produce commands (like ‘Http.send’ or ‘WebSocket.send’ provided by effect manager libraries). That’s a lot different from taking a Cmd value and making expectations about it, as the API @rtfeldman describes above does.

On the other point @rtfeldman makes: I said I want my tests to know as little about implementation details as possible, but that’s not to say that they should know nothing about the unit under test. How could they? My tests at least need to know how to exercise that unit in code, after all. In general, unit tests need to know this much: the public interface of the unit I’m testing (it’s exposed functions) and the public interfaces of those other units it depends upon. The implementation details I don’t want my tests to know about are the choices I make ‘behind’ the unit under test’s public interface to give it the behavior I’ve described with my tests.

When I’m writing a test with Elmer, I’m describing the behavior of a unit (usually a model, view function, and update function working together) in isolation from the other units it depends upon (like the elm-lang/http module). To accomplish that, during the test, I’ll need to provide stand-ins (test doubles) for any other collaborating units. A spy is just a special kind of test double that records calls made to it. I’ll use spies to help me describe and verify the behavior of the unit under test with respect to those other units it depends upon. Writing good tests involves making good (sometimes difficult) decisions about where and how to draw the lines between units, what to test about those interfaces, and so on. For more on this, see post 8 above, where I addressed a similar point.

As you noted earlier, “I want my tests to know as little about implementation details as possible” - and it is possible to completely exercise the behavior of functions using unit tests without knowing the public interfaces of other functions they depend on.

Certainly unit tests need to know the public interface of the unit they’re testing.

Why should they need to know the public interfaces of the other functions they depend on? It’s an implementation detail that’s not required to completely test their behavior.

Let me give you two examples to hopefully clarify my point:

Suppose you’re testing a particular function: myFunction: CoolRecord -> AwesomeRecord -> String. In order to test this function you’ll need to pass in a CoolRecord and an AwesomeRecord and you’ll need to know how to construct these. When I say that a unit test needs to be aware of the dependencies of the unit under test, I’m thinking that CoolRecord and AwesomeRecord are things the test would need to know about. Now myFunction could call all sorts of functions in the process of generating a String given a CoolRecord and an AwesomeRecord. My test doesn’t need to know about those functions called by myFunction. Like I said above, the test only needs to know about other units that are dependencies of the unit under test.

Elmer allows me to adopt a higher-level testing strategy, where I can often treat ‘the unit under test’ as some unit of application behavior that a user would encounter. So, if I’m writing an RSS Feed reader, I might structure my tests around ‘viewing a list of feeds’, ‘reading a particular feed’, ‘reading a particular feed item’ etc, with particular tests describing the expected behavior in each of those cases. In order to write these tests, Elmer needs to know about an initial model, the view function, and update function of my app. I’ll then need to do some stuff in my test setup to get me to the right place of the app to exercise the behavior (click on a button or input text etc). Then I can make expectations about the behavior. However, in most cases, there are additional dependencies that need to be in place to simulate the behavior I want to describe. For example, if I’m writing a test that describes the behavior of viewing a list of feeds, I’ll need to stub the http request that fetches the list of feeds. I do that by treating Http.send as a dependency that I need to provide, because (1) I don’t want to make http requests during my unit tests and (2) Elmer doesn’t know how to process Http.send. Instead, with Elmer I can inject a fake command to replace Http.send during the test (there are various strategies to do this which I’ve talked about above), which allows me to make sure the proper request is being sent (that’s part of my application’s behavior when a user attempts to view a list of feeds) and which allows me to stub various return values (successes, errors) so I can describe the expected behavior under various conditions.

Given this kind of test setup, I have tons of freedom to refactor my code and to experiment with different approaches for structuring the app. My tests do expect there to be an http request made, and they need a reference to a default model, a view function, and an update function. But that’s about it; the tests don’t need to know about any other functions called. It’s been fun to learn Elm alongside writing tests with Elmer, because as I learn more about how to structure Elm apps, I can refactor the code and just re-run my tests to be confident my app still has all the behavior I expect.

Sure! My only point is that spies shouldn’t be one of them, because the alternatives lead to strictly better tests.

I think we’ve now each made our positions clear, and we’ve gone several posts without making any progress, so I don’t suppose there’s much point in belaboring the thread. :smile:

Thanks for politely engaging on this topic! :heart:

I encourage people to keep an open mind. I know as I’ve learned more about TDD and testing strategies in general, I’ve found that test doubles, including spies, can be used in both bad ways and good. When used correctly, test doubles can make some kinds of tests easier to write, and help you test elements of a software system in isolation.

I’m happy so far with the kinds of tests I can write today with Elmer, but I would be happier still if better approaches present themselves in the future.