When do we need binary decoder and encoder?

[jinjor]

I’m very excited to see the elm/bytes repository which was made public just several hours ago by Evan.

GitHub - elm/bytes: Work with bytes and implement network protocols

Please read the README first, before you read the rest I’m going to write. It describes what this library is for and what it is not for.

A use case

So, I’d like to answer the following question written at the bottom of the readme.

If you have a concrete scenario in which you want to interpret bytes as densely packed arrays of integers or floats, please describe it on https://discourse.elm-lang.org/ in a friendly and high-level way. What is the project about? What do densely packed arrays do for that project? Is it about perf? What kind of algorithms are you using? Etc.

I have a very concrete use case. It is about the MIDI data format.

MIDI stands for Musical Instruments Digital Interface. It is (1) a binary protocol for transfering signals between instruments and other devices, and also (2) a binary file format to record musical information like sequence of notes, tones, tempo, etc. I think these data is not necessarily a binary, but it is already used widely and I don’t know any alternative that have a text format.

To make nice music apps, we need to read and write MIDI.

• Read MIDI file (SMF) that is uploaded by user, and show the data
• Output a new MIDI file after some edition
• Realtime playing / recording via Web MIDI

All of these need decoders and encoders for bytes.

Example

For an example, here is a MIDI (+MP3) player I made.

This is written in Elm 0.18 and uses some native code to decode MIDI. I was surprised that this app can also be embedded in tweets. So I can say “hey, this is an app written in Elm!” but sadly it requires some hacky way for now.

If you are interested in this app: my homepage

Other use cases?

I don’t know if there are any other variable use cases. I agree with the last statement in readme. Mutating large binary data seems hard in Elm and needs a lot of investigation to make a decision. Instead, it should be eadier to get some metadata from binary data, like size of images, length of audio, etc.

Anyway, I think this is a good chance to collect all possible use cases we have. Any ideas?

Note: Things like file upload (which many folks want) is also discussed in other thread: Binary and file handling in Elm - #7 by norpan

[ldesgoui]

It’s exciting to see work being done in the music space, I’ve been interested in live looping/live coding recently, best of luck with elm/bytes etc

[cmditch]

I suspect things like elm-bigint and elm-keccak could be refactored to become much more performant using bytes. Crypto libraries in general will start to become feasible.

[wbrian]

This isn’t a project I’m actively working on, but this certainly would have been useful when I was:

I implemented a remote debugger for a NES emulator I wrote. An important aspect of a debugger is to be able to explore the address space, and whenever the debugger broke execution it would send the 64KiB address space over in a json payload.

Anyway, sending binary data in a JSON payload is really inefficient, and I did quite a bit to make it more efficient, such as storing multiple bytes per value and teasing them out before simply writing a native module that wrapped typed arrays. Even this wasn’t an ideal solution, since the address space was sent over in a base64 encoded string and then converted to a typed-array natively.

[billstclair]

Well, crypto libraries are feasible, just a bit of a pain.

https://package.elm-lang.org/packages/billstclair/elm-crypto-aes/latest (updated to 0.19)
https://package.elm-lang.org/packages/billstclair/elm-crypto-string/latest (not yet updated to 0.19)

Public key crypto is basically just bigints over a prime number field. More performant bigints would help a little, but you mostly use public key crypto to encrypt symmetric keys (e.g. for AES), so it doesn’t need to be fast.

[jinjor]

Thanks for your replies!

@wbrian
Very interesting. I don’t know about NES, but it seems a good use case we need to read directly the binary.

Any other cases are still welcome. Tell us what you know.

[unsoundscapes]

I can think of two more use cases related to WebGL:

1. decoding glTF format to import meshes & scenes that were created in 3d software
2. decoding binary font format (TrueType seems to be the easiest) in order to render text with WebGL

[s6o]

I have this little project waiting for a proper solution: https://github.com/s6o/elm-msgpack

From elm/bytes README, a more performant implementation with proper http and websocket support would look to be possible.

[jinjor]

Thank you for giving more information!

So I tried to categorize the use cases.

• serialize arbitrary data
  • this is the original purpose of elm/bytes
  • protobuf, MessagePack, etc.
• read / write small data with binary format
  • this is the main topic here (which I believe nice to have)
  • decode whole data from binary, do anything in the Elm world and encode it to the binary
  • take large data but read only small chunk of data (like meta data)
  • MIDI, glTF, TrueType, etc.
• directly edit the large mutable array
  • this is out of the scope here, needs more discussion about the concrete use cases
  • images, sounds, movies, etc.

The third one seems very difficult at the moment, but the second case is easily done with current elm/bytes implementation. So I could not find any downside when elm/bytes is used for this purpose. Each case is niche but very attractive. I hope that it won’t keep private or internal forever.

Any other wonderful cases you know is welcome until when this thread ends.

(I’m still wondering where to go from discussion. Saying like “Hey Evan, here is a report.” or making another implementation as elm-explorations? What’s the right path? In the context of explorations, I confirmed my jinjor/elm-binary-decoder worked well for me at least.)

[evancz]

It’s not going to be unpublished forever! This week is elm-conf and StrangeLoop, so there are lots of things going on that require shuffling around some of the work related to making sure bytes will fit well with all the various APIs that may need integrations.

[jinjor]

Cool, thank you! I hope you got some inspiration to make this package nicer