I might be wrong but it feels like the error displayed from elm-test when an Expect.equal fails, is a string diff of the result of running Debug.toString on the two values being compared.
This is more than adequate for simple values but for nested mixes of records & lists it feels less helpful and sometimes quiet a strain to read in the terminal output.
I was wondering if there is a chance of using a similar approach to the Debug.toString but instead converting to a generic data representation, perhaps something like this:
And then diffing the two results in Elm with a clearer display closer to the diff one might get from modern JS testing frameworks.
It assumes that all data is ultimately translatable to such a structure and I don’t know if that is true. You might notice I haven’t included a representation for custom types though I feel it could be possible.
I tried to experiment with the concept a little but it involves Kernel code and the changes I’ve attempted in my local clone of elm-explorations/test result in a Corrupt Dependency error when running the ./tests/test.sh script and I’ve no idea how to progress.
Do you think this idea has merit? I suspect that if it did it would have been tried already but I’m curious all the same.
I would love to see diffs that do not use Debug.toString. Firstly, they will look nicer and the code would be less whacky. Secondly, it would be nice (in the long term) to he able to run tests on optimised compiles which is only possible if the dependency on Debug.toString is dropped.
The code that (currently) does the diffing is the test runner (https://github.com/rtfeldman/node-test-runner) rather than elm-explorations/elm. (I think?) It would need to be moved into the elm package if it were to use kernel code.
Can you share examples of the kinds of thing you’re looking to improve? I’ve had issues with the diff highlighting too, but I wonder if we’re jumping to a solution too early here? What would you want diffs to look like under the new scheme?
Thank you both for the responses. I’ll work up some examples to better illustrate it and, yeah, good point, there might be better solutions if one is needed.
A kind commenter on Slack noted that I was really asking for structured diffs over textual diffs. I guess structured diff-ing is a specific concept and approach that makes sense in these situations.
I’ve done a small amount of research with a comparison to Jest in the JS world that seems to do more structured diffs. Here is a simple jest example:
And here is a similar arrangement for elm:
Here the experiences are similar though not the identical, which is fine.
I’ve then attempted to create a slightly more ‘worst case’ scenario for the text diffing approach by having names & values which include Just & Nothing. It is hard to do a direct comparison due to the nature of the two languages’ data representations, but here is a Jest diff:
I feel like I have experienced quite unpleasant diffs in the wild on production code where the result feels like a random patchwork of highlighted characters. I will attempt to share such an example the next time I experience it.
I’ve been messing around with the initial idea above of doing a diff once the data structures have been translated into some kind of generic representation in Elm. I don’t know if it is possible to pursue this line but if anyone is interested my efforts are here:
They aren’t based on any research so it might be flawed and the output is far from perfect at the moment but it has been fun to try to think through the problem a bit.
