Typed Arrays for Elm

[szabba]

@rupert

I think such a library would need to be implemented in native code. For example, if multiplying 2 matrices, the result will be written to the output matrix using Buffer.set many times if it were done in Elm, and Buffer.set is going to copy the whole Buffer. A native implementation would read from the 2 input matrices, and build up a the answer in a mutable array, then return that as a Buffer back to the Elm program.

A function like Buffer.initialize : Int -> (Int -> a) -> Buffer a would be a good building block for these sort of operations. Array.initialize is already a thing.

[mattpiz]

@mzero

I hope to have some more detailed response to the points in this thread later… But for starters:

While I’d love to have people try this, now that we know that native code will not be usable in 0.19, I recommend that you don’t spend time doing so. But one thing that will still be very valuable is feedback on your concrete use case. In your second case for example, you mention slicing and joining. Great, I’d suggest that once you’re advanced enough in your project, you create a post explaining the API you’ve come up with, and the pain points you had implementing it with current available array API and ports. This will be very valuable input for core elm members. And possibly, some solutions might appear in pure elm for part of these issues.

@mzero

Today, I bridge via port from the JS WebAPIs that deal in UInt8 arrays, and elm’s Array of Int.

I used to do that also for one of my image manipulation use cases. Eventually I had issues with elm Array type that forced me to switch to Robin’s array exploration (HAMT array). At this moment, JS interop by ports became a burden because HAMT array was not a type directly supported. The good news is that Robin’s work is now merged in elm 0.19 so it will get better in 0.19.

@szabba

A function like Buffer.initialize : Int -> (Int -> a) -> Buffer a would be a good building block for these sort of operations. Array.initialize is already a thing.

Indeed

[Warry]

Great work Matthieu ! And very detailed explanation. Like you said, Typed Arrays are everywhere in modern browsers. at Nomalab we use Blobs (File api) to compute large files’ md5 (hundreds of GB), before slicing, encrypting and sending them to AWS S3. Right now this is is done by a quite buggy js library, but at some point we will have to do it ourselves in order to improve resuming after errors. It’d be awesome if we could do it in 100% Elm.

[Robert]

Nice benchmarking work, I need to do this myself.

I have gone and done the opposite and removed typed arrays from linear-algebra https://github.com/robert-wallis/elm-linear-algebra/tree/PureElm?files=1 don’t shoot me, I am just experimenting.

What I want to know is practically, when dealing with arrays of length 2,3,4 and 16 if typed arrays help or hinder performance.

Also, unrelated to linear algebra, just yesterday I discovered that new Float32Array([1,2,3]).map(()=>”hi”) returns [NaN, NaN, NaN]` so the elm compiler cannot use native map to implement elm map.

[Robert]

And by practially, I mean linear algebra operations like Mat4 multiplication in elm. I’m guessing streaming a large float array to the gpu is faster, which is also practial.

[mattpiz]

@Robert

I have gone and done the opposite and removed typed arrays from linear-algebra elm-linear-algebra/ at PureElm · robert-wallis/elm-linear-algebra · GitHub don’t shoot me, I am just experimenting.

Thanks @Robert, haha don’t worry I don’t “byte” . My approach is essential generic about typed arrays. Your’s makes me think of the one of @zinggi by the way (Zinggi/elm-webgl-math) except he used tupples where you used records. I’d encourage you to do benchmarks to evaluate all this. elm-benchmark is really nice to work with.

@Robert

Also, unrelated to linear algebra, just yesterday I discovered that new Float32Array([1,2,3]).map(()=>”hi”) returns [NaN, NaN, NaN]` so the elm compiler cannot use native map to implement elm map.

Regarding that, it is not actually an issue in elm. In the API I have:

map : (b -> b) -> JsTypedArray a b -> JsTypedArray a b

So you can only provide an Int -> Int or a Float -> Float as a mapped function. Elm prevents these kind of things from happening at compile time. But there is still a reason not to use map in the implementation, and that’s browser compatibility.