Check if Elm has things in the message queue

[berend]

In order for Behat (Selenium) tests to work better, it’s handy to know if Elm still has stuff in the message queue.

For example after I click on a button, I usually cannot immediately check if some text on the screen has updated. There may be some delay, either by Elm still doing calculations, or waiting for HTTP response.

It’s annoying to liberally sprinkle sleeps in a test. It would be much better if I could detect both conditions.

Does anyone know how, from JavaScript, I can check if Elm’s message queue is empty? I.e. Elm is in “idle” mode.

Secondly, is there also some queue where Elm waits for the result of an http request?

[Herteby]

Instead of doing something like that, I would suggest writing all your test commands and assertions as retry-until-successful. That’s how for example cypress.io works. I believe Selenium has some built-in feature for that too.

It might sound sloppy but it’s actually pretty good. Here’s cypress.io’s page about it: https://docs.cypress.io/guides/core-concepts/retry-ability.html

[genthaler]

I heartily agree with @Herteby, retry is better. In fact I’d go further and say that adding delays to a test suite is death by a thousand cuts. I don’t have any experience with Behat, or indeed PHP; if you were using Java I’d suggest using Serenity. This SO post recommends using a spin function that might help.

[berend]

retry-until-successful

Well, but how do you catch errors then? Obviously by still building in a timeout. The other drawback is that you may have to wait very long for a test failure which should be pretty obvious.

I agree that retry until successful is a good approach when waiting for http request return, but doing that for ordinary screen updates seems I would have to wait quite long for the test to determine something isn’t working.

How big is your timeout?

PS: will have a look at cypress.io.

[Herteby]

I think for CI we currently have it set at 20 seconds (we have a few reallyslow endpoints, and it’s nice to have a big margin). But it’s not a big deal since we only pay that cost on failures. While writing/running tests locally you can set a lower timeout ofc.

[reitzig]

I haven’t done any amount of serious Elm development, but this has the feeling of a non-issue. The kind of error you’re thinking off – some message not sent correctly – would be caught in unit testing in Elm, wouldn’t it? By the time you get to acceptance testing using Selenium, that part is already fine. Am I missing something?

That said, when testing Elm an empty message queue is (sometimes) a clear indicator of “computation done”, so to speak. It could be a good addition to timeouts: “wait 20s or until message queue empty”. Failing tests faster is always good.

[berend]

Am I missing something?

Yes. Some Behat frameworks peek into the underlying software to see if it’s still doing things, I wanted to do something similar.

But have followed advice here and used the retry method. Works, and probably more reliable.

[nateabele]

Instead of doing something like that, I would suggest writing all your test commands and assertions as retry-until-successful.

Polling is only ever a workaround for trying to integrate with a black box system. It should almost never be used if there are alternatives available.

That’s how for example cypress.io works.

Cypress works that way because it’s universal/agnostic, and they go to great lengths with tooling and APIs for writing mock boilerplate that should be superfluous for a programming environment like Elm.

To this day, Karma is the most reliable E2E framework I’ve ever used (and I currently use Cypress in production extensively), specifically because it was tightly integrated with Angular and could track its internal state. @berend’s intuition is correct here.