Using elm-html-test effectively

Hi! In my Elm project, I recently implemented pagination. I ended up defining a “widget” (i.e. view helper/utility function) that looks something like this:

Screenshot%20from%202019-10-21%2020-21-59

To render, this “widget” has a bunch of logic to figure out how many pages there are, how many page numbers to show, whether the “Previous” and “Next” buttons are active, whether to put the “…” ellipsis anywhere, etc. It was sufficiently complicated that after I wrote it, I thought I would try testing it, which I attempted to do with the HTML testing functions in elm-explorations/test. I found it very difficult to write tests that expressed the kinds of invariants I had in mind. I thought I would ask if I was missing something obvious or if there’s some way to write tests that I hadn’t considered.

Here are the tests I wanted to write and what I eventually did to try to implement them.

Test 1: there should generally be 9 page numbers you can click on

For this test, I wanted to render the pagination widget with 100 pages and a random page number and ensure that the “window” of page numbers around the current page remains constant. It was easy to figure out how to find only links, but the “previous”/“next” links don’t really count for this test. I couldn’t find any way to exclude them – all selectors are positive and I didn’t see a “not” or “drop” – so I ended up adding a class to the rendered HTML for page numbers, so I could findAll the elements with the page-number class. This feels a little dirty, because these classes will end up in my optimized build but only exist to facilitate testing.

Once I had that, I was faced with the problem that the first page/last page buttons are also rendered as numbers, so have the same classes. I could have added another class to distinguish “window” numbers from the first page/last page, but this was getting complicated enough. I settled for writing the test to ensure that there are either 10 or 11 numbers (you will always see at least one, and possibly both, of the first/last buttons).

Test 2: there should never be any ellipses between consecutive numbers

For this test, I re-used the same 100-page widget with a random “current page” idea. It was harder to figure out how to write the test itself. Finding an ellipsis is easy, but then using that to get the numbers on either side is challenging. It’s possible to get an element for a specific index, but it’s not possible to get the index of the ellipsis – you’d have to encode out-of-band knowledge about where to find it, which makes the test more brittle.

Even if you could knew which indexes were before and after the ellipsis, there aren’t any real ways to extract information from the HTML. There’s no way to say “Get the text, parse it and see if it’s the same as the other number +/- 1”. You can check for specific values (2 for the beginning, or 99 on the other side), but even that isn’t super helpful.

An alternative approach might be something like “Ensure that there is either the ‘first page’ button and an ellipsis but no 2, or instead that there is the ‘first page button’ and a 2 but no ellipsis”. But I couldn’t figure out how to do this either; there’s no Expect.any (only Expect.all), so you can’t check for “one of these cases”. There is an Expect.onFail, but you can’t use it to recover from failure, just to provide a different error message.

In the end, I abandoned writing Quickcheck-style tests for this. I handwrote test cases for page=5, page=6, and page=7 and wrote assertions about what I expected to find in each case. This isn’t great because it relies on knowledge of the boundary conditions and edge cases of the pagination widget. Doing the same thing for the ellipsis at the other end of the widget seemed like too much work so I didn’t bother (hopefully all my bugs are symmetric).

Missing links

I think I could have written more effective tests if I had had access to one of:

  • some kind of DOM navigation: you can navigate to the children of a node, but not back to the parents, nor forward/backward to sibling nodes
  • extracting information from Querys: get the text contents, convert a Multiple into a List Single, find the index of a thing. There’s each but there’s nothing like a fold that would let you carry information from node to node. There’s has and hasNot, but they produce Expectations, which are hard to work with.
  • a not selector
  • when using Expectations is necessary, more Boolean operators on them

Instead, I had to rely on knowledge of the internals of the code under test, and hard-coded results.

Is any of this kind of functionality on the roadmap for elm-explorations/test? Based on the name, I assume it’s still being defined. Or am I maybe using it wrong, or is there something unique and unusual about my use case? Thanks!

There are several issue about some of these things. Though a lot of these issues need to be migrated to the new github project.

See Some way to find an element labelled by another · Issue #52 · eeue56/elm-html-test · GitHub

See Have a way to extract an attribute on a found element · Issue #51 · eeue56/elm-html-test · GitHub regarding attributes. Extracting text is maybe a new one, so perhaps create a new issue for that (on elm-explorations/test, not on eeue56/elm-html-test) – though that gets closer to the line I think we want to avoid crossing of making elm-html-test a general library for inspecting HTML.

Note also that there’s an existing workaround I often end up using of using Test.Runner.getFailureReason to determine if an Expectation failed or not.

One issue here is that the current Selectors have some inconsistencies that cause them to do unexpected things in certain cases (for example: Query.has matches children, not just root element · Issue #41 · eeue56/elm-html-test · GitHub, `Selector.containing` matches text anywhere in the `Single`, instead of only matching the candidate element · Issue #67 · eeue56/elm-html-test · GitHub). I suspect that implementing not will probably expose more weirdness in this area, and that probably the Selectors need to be reworked with a more solid logical foundation to really solve some of the missing features.

See Selector.oneOf · Issue #49 · eeue56/elm-html-test · GitHub


Currently elm-explorations/test doesn’t have anyone actively developing new features (at least for the Html.Test stuff – there is some active work on Fuzz). Progress in any of these area will require someone stepping up to tackle some of the work and take it all the way through design to a consistent implementation :slight_smile: If anyone’s interesting in contributing, the place to start would be opening an issue on the github repo explaining what you intend to work on and detailing your initial plan.

And to give an answer in a slightly different direction:

One style of using quickcheck is to compare the actual system to a simplified model. I think that kind of approach would probably solve the issues you’re describing.

It sounds like you’ve been trying to write properties saying that “for any random input, ensure that all adjacent pairs of buttons differ by exactly 1”. As you’ve noted, current limitations of elm-html-test prevent you from inspecting the HTML in a way that allows you to do that.

To use an approach of comparing to a simplified model, you could instead write a function that takes the random input and produces the expected list of buttons. That function is your simplified model, and your real view function is the real system. Your fuzz test would then take a random input, use the simplified model to get the list of expected buttons, and then assert that all the expected buttons are present in the Html produced by the real view.

1 Like

Thanks! I’ve subscribed to a couple of these issues.

Point well taken about the need for more contributors on this project. I’ve been hesitant to try contributing to any of the Elm projects so far, thanks for explaining what shape that contribution would need to have in order to be appreciated. I don’t know if/when I will have time to try anything here but I would like to.

I also considered the idea of having a simpler function that produces some kind of abstracted pagination object and running tests against that, but the presence of the ellipsis elements made it feel like I would be implementing something very HTML-like. But thinking about it again, maybe I could come up with something that would cover most of my use case. I’ll give it a shot. Thanks again!

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.