Not exposing internals hurts testability

First of all, thanks for the advice. The reverse testing was put away in my head and it’s a good refreshment :slight_smile:

I agree. That’s also what I’m trying to do. I guess it’s best to share an example to express my challenge more clearly. So this is basically the structure I use:

import OtherModule exposing (OtherType)

type alias Model = { a: String, b: Int }

type SomeType = 
     SomeType Model

doCmd : SomeType -> Cmd msg
doCmd (SomeType someType) = 
    -- Do something with someType

parse : OtherType -> SomeType
parse : otherType
    -- Transform OtherType into SomeType

In this case I would like to expose only ‘SomeType’ and the methods ‘doCmd’ and ‘parse’ to force every logic arround SomeType to take place in this module

The challenge I have, there’s no way of testing the ‘parse’ method because I cannot construct a SomeType for an expected result outside this module.

Does this make my challenge more clear?

This is something I also thought of. I must admit I don’t like it, because I see testing and implementation as seperate things and I would like to keep them apart.

I hope there’s another solution.

1 Like

I often encounter this need when I’m trying to nail down a complex function. Usually, I have an idea about how to split it into simpler parts, and I start by writing those simpler parts. The problem arises when I try the TDD approach. I need to write tests first, but the complex function is nowhere near ready yet, and the simpler function isn’t supposed to be a part of the exposed interface. So what am I supposed to write tests for?

I begin by exposing the internal function and writing tests for it. Then I get those tests passing, do the refactoring etc. à la usual TDD. I rinse and repeat for other simple parts of the complex function, and then I compose the actual exposed complex function from those parts, again using TDD.

The trick at that last step is reusing the tests for the internal parts to test the top level function instead. If it defines a promise of the final API, it gets converted. If it just tests the internal interface, it goes away, having served its purpose.

Then I’m left with just the tests for the final top-level complex function, which hopefully cover most of the cases it’s expected to work in. If there are any more cases left to cover, I write them and tweak the code as needed to get them passing.

At that point, I can safely un-expose the internal functions and just test the public API. Even if the API needs to be upgraded later, there’s usually no need to expose the internals again, as I already have working code. Even new internal functions usually don’t have to be exposed, as their contribution can be tested through the working public API.

I see this approach as a sort of work-in-progress “scaffolding”, to be removed once the API can stand on its own feet.

5 Likes

To be sure I understand you: Taking above example, in the beginnning you would expose Model, and test the ‘parse’ function. If it works you would throw the test away and hide Model again?

Yes, I think. Which part of this would be the final exposed API?

Testing language-level opaque structures like Cmd is another pain point entirely :slight_smile:

Since parse isn’t something a user of your module can use in a meaningful way on it’s own, you can’t test it in isolation.

Only the composition of parse >> doCmd (with type OtherType -> Cmd msg) is meaningful for a user, so that’s what you should test.

In your case this is made a bit more difficult by the fact that doCmd returns a Cmd msg, which isn’t really testable. You might consider returning something else, e.g. an exposed custom type that stands for all the possible cases. E.g. something like type OutCmd = Cmd1 | Cmd2 | ...

1 Like

<rant/intrested-in-this>: As someone not into TDD but agreeing on its promise this is always what gets me. If I can only test public API I can only have tests for a module returning “Some expected test result” for a given input but if I want to test the actual code I would like tests for the functions involved. Now I can only test the whole not the details. As in I dont want my algorithm to be publicly used but I would like to make sure my algorithm works as I expect it to. If only allowed to test from the highest level there are so much more that can break/make it harder to debug than just call that function and verify it does the right thing. It sort of feels as the case for unit tests vs. integration tests. As in “integration tests are worthless, you get no info on what went wrong and they are slow/have too many concerns”. I feel you are sort of in integration test mode when only testing public API (for a complicated module) instead of writing a test for calculateFractalAnomaly: Int -> Float -> Array Byte you need to setup the dependant thing generating the `Array Byte´ and do a whole dance around converting strings to the data and so on. When developing should you write tests for the internal functions and throw those away when you are done or what? I saw some post a while ago where you could add the tests to your module which allows you to test internal stuff. My immediate reaction was that this is the correct way. What if Elm would have that feature in core?

1 Like

What you should do is write tests using the public interface with setups that would ensure each branch in the internal implementation is reached.

These are still unit tests, integration tests are at a higher level. The unit in this case is the module and its exposed interface.

1 Like

I get that. But the thing discussed is that that is not always reasonable right?

What I meant with throwing away (and the comparison to integration tests) was that starting a project you start with the internal stuff maybe and you do it using TDD. Later you arrive at what should be public api and then you have clear tests that you need to rewrite using your public api but that means you need to pull in other stuff/make it more involved just to setup the tests since the public api takes xml-data for example so now you need an xml library to test your thing and you end up further from the actual thing you are testing and add more points of failure. Bad example I know but trying to make the point. :slight_smile: In a lot of cases I feel it would be valuable to keep those tests that are clear and to the point.

Is this something that is considered a big NONO in TDD community or are there different camps?

I have in the past worked with obj-c/swift and there you can expose stuff only for the test-target.

Something like this in Elm maybe:

module Mikaxyz.UiWidget exposing 
    (view
    , Attributes(..)
    , someFunction for Tests
    , SomeType for Tests
    )
1 Like

I just think there is yet no clean and elm-like way in elm, to do tdd. I like your idea of exposing internals for tests though :+1:t2:

I think testing the public interface is okay and sufficient so far, but if you want complete coverage, or develop the tdd-way from the very beginning of each module (which I also wanted to do at some point and realized, it’s not possible ) you get stuck.

Here you have two interesting talks, the second one approaches testing very differently. So maybe this edge is a point where Elm could develop a nice solution for itself and it’s tooling…

Writing testable Elm:

Creating tests from types:

I like it, that is a really smart idea - turning the internal function tests into part of the overall tests for the exposed functions.

I also like your approach because it sits well with how TDD is intended to be done. You write tests in order to help you develop the code, and not as an after thought to test code you have already written. I think that is the thing about TDD that a lot of people miss - they think its primary purpose is to test code, when in fact its primary purpose is to guide the development of code through testing. A side effect is that you end up with testable and well tested code.

2 Likes

I think he/she wants to test the smallest testable unit possible (which are functions). Therefore testing the Modules public functions is indeed more like an integration test, since you are not testing the Module as a unit (you can’t), but its exposed functions (those are the units in this case - and we are back to the smallest testable unit again). Testing all the Modules units is the point I think.

1 Like

I think this is a good idea by itself, but nor for beginners or even experienced folks (it is not straightforward and not easy to understand). Also, you will loose your internal tests if I understand this correctly. What if you want to refactor those internal functionalities in the future, with the aid of your tests (which are gone at this point) ? I think one big plus of Elm is, it’s ease of refactor. However, even there you can make mistakes. semantically.

I just think on one hand Elm makes refactoring easy and reliable, but on the other hand, it shouldn’t forget that testing all the small units is also important…

1 Like

The internal tests will still be there, just evolved to be sub-parts of the tests for the exposed API. So you can either refactor those internal test functions with the code, or if the refactoring is more major, I would switch back to exposing (..) and build up some new internal tests functions and then re-incorporate them into the public API tests as you go.

Hm to be frank, that sounds complicated, complex and prone to bugs.

I lean towards only testing what’s exposed. If something is tricky enough that it needs a test, and it’s not exposed, it tends to fit better in a different module that exposes it. Contrived example: if I have some tricky logic that includes some date calculation, and I want to test the actual date calculation too, then I extract it to e.g. a DateCalculations module.

I’ve done TDD across many languages and I can’t think of a feature that Elm’s lacking in this area. As other’s have mentioned TDD forces you to decide where your boundaries are and what behaviour to expose. If you find that there’s an aspect of your internal implementation that you wish to test in more detail it may be a sign that you should extract that behaviour into a separate more focused module and write tests against that.

5 Likes

What if the only exposed function is view and variants of that? That means you will most likely have to test for some outcome of Html msg. (Does elm-test even do that out of the box?) I have a feeling there are a lot of opinions here that are based on a certain use case. Is it a package? An app? Since this question/concern is reappearing throughout the years perhaps there are stuff that causes friction for users. Dismissing it by saying this is fine because I have never had a problem with it might not contribute a lot to a solution…

Not intended to be hostile in any way. Just trying to expand the discussion and hopefully expose solutions to the problem… :sunglasses:

You said “extract into separate module” and that is basically what I wish to do but I don’t want my users to use that module since that is implementation details. But I still want to test it… Am I missing something here?

2 Likes

I would argue that you are sticking to a technique that doesn’t make much sense for Elm, just because other languages use it.

TDD was formulated around the idea that your tests should test everything at almost every level of writing code, both unit and integration tests, and hence reduce the amount of bugs.

Although it sounds sexy it is hard work, a lot of before-hand thinking making sure that you can move trough pastures of green check marks. But if you take a step back, I believe that valid question is how much TDD is worth in Elm? I would argue that testing everything doesn’t make sense in this ecosystem, and it is not worth the effort, i.e. you are not getting you bang for a buck.

Having said that, you still need to test stuff, especially when you are developing parsers, and such data wranglers, you would like to make sure that they work against squad of test cases.

But does it make sense to hide types needed to test then? Especially if you are developing an application, hence exposing some gory details of your package won’t hurt anybody, hence there are nobody using that module except application developers i.e. you.

So, my two cents are:
Relax, compiler does much of the work, and you write a lot to satisfy that guy, there is no need of inventing new hoops that you need to jump trough, just so you can’t see implementation details of your own module. :slight_smile: Hence expose what ever you want to test, it might not be sexy, but at the end of the day, working code is much nicer than theoretically perfect module structure. I have wrote hundreds of thousands of lines of Elm, deleted many many thousands of those lines, because of unnecessary bureaucracy that I invented for my self, because that is what I would do in [obj :C]

Keep it simple <3

2 Likes

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.