WebGL.Textures.fromList

Request Feedback
1

WebGL is one of the main reasons why I start learning elm at all. After 2.5y of investigation found that elm is the easiest way to do some crazy experiments. But also found that there is a lack of main functionality that is provided by WebGL, especially in textures, and 2D filters (Yes WebGL is not only about 3D).

Images are great - but that is not the only way Textures can be used for, but Elm implementation is limited to only that, and most worse part of that - you cannot get Texture without Task (that is due to JavaScript limitation of loading image in an async way, even from base64 encode url).

So I started the investigation, how at all WebGL works in elmā€¦ and found that hacking on elm-exploration stuff is only for ā€œchosen oneā€ - as the development process is kept as The Secret, no manuals, no documentation, no comments in code, code is formatted in a weird way, no generic tools used to format elm or js code.

But after some tweaks/hacks/manipulations I finally got it working (feed in my changes of native code to elm). But letā€™s keep that as The Secret.

Back to Textures - they (textures) can be used not just as images, but also as 2d arrays with some data (each pixel is just 32bit - where you can encode any data)

Would be nice to have access to that, and be able to compose Texture in elm, that would give to advanced developer more freedom, and beginners more gradual steps to evolve (not think about Task, and require upgrade Browser.sandbox to Browser.element).

Now We have:
WebGL.Texture.loadWith : Options -> String -> Task Error Texture

I was able to create new function to creating texture:
WebGL.Texture.fromList: Options -> Width -> List Pixels -> Texture
*Pixel is Int in format 0xRRGGBBAA

Use cases:

  1. You can create texture without Task, and use it as the fallback or initial value for textures, so no need to create different models when a texture is loaded and when not.
  2. Texture can store permanent data - cracks on walls, bullet holes, dead bodies - something that updates rarely, but need be passed to the GPU somehow
  3. Randomness in shaders - can generate noise texture, and use that in the shader, instead of calculating noise values each frame per each pixel
  4. Generate Look Up Tables - for tile-based games ( More details about this rendering technique can be found in Brandon Jonesā€™ blog.)
  5. Testing - texture can be create as mock in Model (to test states with loaded textures)
  6. Custom image or/and scene loaders - the community will be able to create custom format parsers that can be loaded in WebGL (images: TGA, DDS, BMfontā€¦ scene: gltf, BSP)
  7. Any shader that requires a lot of data and data can be obtained only during runtime.

Problem with my current solution - is that it hides some stuff underhood:

  1. List Pixel can be any length, but the image must be rectangular - so it automatically adds empty pixels (rgba(0,0,0,0))
  2. if a texture is non-power-of-two (NPOT) (and in my opinion that is most of the cases with generated textures) you cannot generate mipmaps, so WebGL.Texture.fromList will silently overwrite those options.
  3. List on Js side is quite not so pleasant to work with, so it requires to loop over all ints, to pass it to the texture binding.

So I see a few solutions to that:

  1. WebGL.Texture.fromList: Options -> Width -> List Pixels -> Result Error Texture
    where Error is just SizeError or Texture
  2. Leave as is - and explain all fallback cases in documentation
  3. Have two versions:
    3.1 WebGL.Texture.fromList: Options -> Width -> Height -> List Pixels -> Result Error Texture
    3.2 WebGL.Texture.unsafaFromList: Options -> Width -> List Pixels -> Texture

Any API / suggestion / objection / questions are welcomeā€¦ (code will fallow as pull request when will find the best solution - hope you will help)

4 Likes
2

Thanks for starting this discussion!

For my use case, I really donā€™t mind the task return type, but having to construct a (List Int) to pass over a port (at the moment, not required in your thing) is more of an issue for me.

My use case is decoding images out of downloaded bytes, and displaying / manipulating them in a WebGL shader. For me, the ability to create a texture from bytes, even via a task, would be fantastic, ala something like:

WebGL.Texture.fromBytes : Options -> Bytes -> Task Error Texture

The upsides to using Bytes over Texture for me are in the debugger (even though the texture isnā€™t currently supported in the debugger), as I think itā€™s unreasonable to ever expect the debugger to ā€˜nicelyā€™ handle massive lists of integers (mine are 1,000,000+ ints, and I expect these to grow significantly.

Itā€™s really important for me to avoid UI hitches, and right now the way I achieve this is via passing the data out over the port as (List Int) and pass to a background webworker, which is already dangerously close to my ~30ms time budget. Given the WebGLContext methods been called arenā€™t expressly super slow when using typed arrays, Iā€™m believe Iā€™m mostly paying a message paying penalty.

3

https://github.com/elm-explorations/webgl/pull/13 i already had that example to build texture from Bytesā€¦ and feeling (as that is not approved) - no one needs it except meā€¦
and best part abut bytes / ints - it can be done in sync
Decoding binary file, extracting bytes to render in webgl - kind of was my original problemā€¦

4

that is not only your issue - ill also have the dream that someday we will be able to pass Bytes over port
but that topic for other package (Json.Decode.bytes: Decoder Bytes / Json.Encode.bytes: Bytes -> Value)

1 Like
5

What about using tortus/elm-array-2d or your very own justgook/elm-image for building the texture?

6

You already can use elm-image to create Texture - but that is slow as hell (btw - that was original reason why I creating elm-mage at all)

7

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.