# Bytes, ports and Uint8Array

**URL:** <https://discourse.elm-lang.org/t/bytes-ports-and-uint8array/3569>\
**Category:** Learn\
**Created:** [April 28, 2019, 1:45pm UTC](https://discourse.elm-lang.org/t/bytes-ports-and-uint8array/3569 "2019-04-28T13:45:53Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![martin-jahn](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.elm-lang.org/martin-jahn/32/2021_2.png) [@martin-jahn](https://discourse.elm-lang.org/u/martin-jahn)\
**Post date:** [April 28, 2019, 1:45pm UTC](https://discourse.elm-lang.org/t/bytes-ports-and-uint8array/3569/1 "2019-04-28T13:45:53Z")

</div>

My goal is to get an image form user encrypt it in the browser and upload it to the server. All of these things by themselves work. The only problem is when I start to put all of this together. When I get the file it’s represented as `File` type. And I want to push it through port to JS where I encrypt it. I want to use `Bytes` for this becasue `Base 64` wastes 30% of space and I don’t want to waste it just because of ports.

My initial idea was to `Base 64` encode it decode it in JS to Uint8Array encrypt it and do the encoding all over again in reverse. But I don’t think that it’s a good idea to do it on files because they are a little bit bigger. The overhead might be acceptable for uploads but I’m not sure if it would be acceptable for processing multiple files when viewing more images on the same page. Is there a better way to make it work?

For encryption I use [Tweet nacl-js](https://github.com/dchest/tweetnacl-js).

---

<div class="post-metadata">

**Author:** ![folkertdev](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.elm-lang.org/folkertdev/32/133_2.png) [@folkertdev](https://discourse.elm-lang.org/u/folkertdev)\
**Post date:** [April 28, 2019, 9:01pm UTC](https://discourse.elm-lang.org/t/bytes-ports-and-uint8array/3569/2 "2019-04-28T21:01:54Z")

</div>

It might be a little nicer (and easier) to decode the `Bytes` as an `Array Int`. Arrays can be sent through ports.

When you consider size waste also consider encoding/decoding performance. To and from base64 encoded string is probably much slower than using an `Array Int` (even if that array is slightly larger).

I think it would be nice to be able to transfer `Bytes` through a port (where it would be a javascript `DataView`), and have seen several people asking for it so far.

We then tie our `Bytes` type to the semantics of javascript `DataView` (possibly forever), so it is not an obviously good idea but would be very practical.

---

<div class="post-metadata">

**Author:** ![martin-jahn](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.elm-lang.org/martin-jahn/32/2021_2.png) [@martin-jahn](https://discourse.elm-lang.org/u/martin-jahn)\
**Post date:** [May 1, 2019, 6:22pm UTC](https://discourse.elm-lang.org/t/bytes-ports-and-uint8array/3569/3 "2019-05-01T18:22:28Z")

</div>

This code does the trick.

```elm
import Bytes exposing (Bytes)
import Bytes.Decode as BDecode

decodeImage : Bytes -> List Int
decodeImage encodedImage =
    let
        listDecode =
            bytesListDecode BDecode.unsignedInt8 (Bytes.width encodedImage)
    in
    Maybe.withDefault [] (BDecode.decode listDecode encodedImage)

bytesListDecode : BDecode.Decoder a -> Int -> BDecode.Decoder (List a)
bytesListDecode decoder len =
    BDecode.loop ( len, [] ) (listStep decoder)

listStep : BDecode.Decoder a -> ( Int, List a ) -> BDecode.Decoder (BDecode.Step ( Int, List a ) (List a))
listStep decoder ( n, xs ) =
    if n <= 0 then
        BDecode.succeed (BDecode.Done xs)

    else
        BDecode.map (\x -> BDecode.Loop ( n - 1, x :: xs )) decoder

```

On the JS side I need to transform it into `Uint8Array` like this: `Uint8Array.from(data);` Unfortunately it is not as fast as I would want it even on my powerful desktop machine.

---

<div class="post-metadata">

**Author:** ![folkertdev](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.elm-lang.org/folkertdev/32/133_2.png) [@folkertdev](https://discourse.elm-lang.org/u/folkertdev)\
**Post date:** [May 1, 2019, 8:02pm UTC](https://discourse.elm-lang.org/t/bytes-ports-and-uint8array/3569/4 "2019-05-01T20:02:24Z")

</div>

Did you run benchmarks? I actually think that would be really valuable to see how much of a degradation it is. Also for context: how big are the arrays you are sending?

RE your code: I think you’re missing a `List.reverse` somewhere, this decoder will reverse all the bytes. In general it might be better to use an `Array` on the elm side instead of `List`

---

<div class="post-metadata">

**Author:** ![martin-jahn](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.elm-lang.org/martin-jahn/32/2021_2.png) [@martin-jahn](https://discourse.elm-lang.org/u/martin-jahn)\
**Post date:** [May 1, 2019, 10:24pm UTC](https://discourse.elm-lang.org/t/bytes-ports-and-uint8array/3569/5 "2019-05-01T22:24:59Z")

</div>

So I gave up. Support for bytes is not as good as I would liked it would be. `File.toUrl` almost as fast as the code above. Even if that means decoding Base64 in JS again. But it is easier to read so I choose this path.

Much bigger problem is decoding of Base64 in Elm. For 2 MB file it takes ridiculous 10s to decode to bytes and send it to the server. I tried both Base64 libraries listed on [Elm Finder](https://www.elmfinder.org/grids/g/base64/) that are able to decode to Bytes. But both are really slow. Until this is fixed I’m not able to do it any other way.

---

<div class="post-metadata">

**Author:** ![antew](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.elm-lang.org/antew/32/2102_2.png) [@antew](https://discourse.elm-lang.org/u/antew)\
**Post date:** [May 2, 2019, 12:52am UTC](https://discourse.elm-lang.org/t/bytes-ports-and-uint8array/3569/6 "2019-05-02T00:52:42Z")

</div>

One way around it could be to make a custom element that gets the file, encrypts it as a new file object, then sends that into Elm through a custom event where you can treat it as a File on the Elm side, that way you don’t have to pay so much is serialization costs

---

<div class="post-metadata">

**Author:** ![martin-jahn](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.elm-lang.org/martin-jahn/32/2021_2.png) [@martin-jahn](https://discourse.elm-lang.org/u/martin-jahn)\
**Post date:** [May 3, 2019, 9:25pm UTC](https://discourse.elm-lang.org/t/bytes-ports-and-uint8array/3569/7 "2019-05-03T21:25:00Z")

</div>

Your advice is great I got it working all the way to the

> [@antew](#):
>
> sends that into Elm through a custom event where you can treat it as a File on the Elm side

I cannot get custom objects through Elm to HTTP file upload. I cannot get that value from JS to a `elm/File`. It would be perfect if I could. The best thing I was able to achieve is array of numbers which I serialize on the backend to a file.

---

<div class="post-metadata">

**Author:** ![antew](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.elm-lang.org/antew/32/2102_2.png) [@antew](https://discourse.elm-lang.org/u/antew)\
**Post date:** [May 9, 2019, 2:04am UTC](https://discourse.elm-lang.org/t/bytes-ports-and-uint8array/3569/8 "2019-05-09T02:04:37Z")

</div>

Sorry for the delayed response, I created an Ellie ([https://ellie-app.com/5v8nRD2Kvwwa1](https://ellie-app.com/5v8nRD2Kvwwa1)) of getting an encrypted `File` through a custom event, some of it is hacky for expedience, but the meat of it is

```auto
node "custom-file"
  [ on "file-selected"
      (Decode.field "detail"
          (Decode.map FileSelected File.decoder)
      )
  ]

```

And a custom element that handles the dirty work

```auto
class CustomFile extends HTMLElement {
  constructor() {
    super();
    this.processFile = this.processFile.bind(this);
  }
  
  processFile(evt) {
    const file = evt.target.files[0];
    const filename = file.name;
    const reader = new FileReader();

    reader.onload = (e) => {
      const data = e.target.result;
      const iv = crypto.getRandomValues(new Uint8Array(16));
  
      crypto.subtle.generateKey({ 'name': 'AES-CBC', 'length': 256 }, false, ['encrypt', 'decrypt'])
        .then(key => crypto.subtle.encrypt({ 'name': 'AES-CBC', iv }, key, data) )
        .then(encrypted => {
          const evt = new CustomEvent("file-selected", {
            detail: new File([encrypted.buffer], filename)
          });
          this.dispatchEvent(evt);
        })
        .catch(console.error);
    }
    
    reader.readAsArrayBuffer(file);   
  }
  
  connectedCallback() {
    const input = document.createElement('input');
    input.type = "file";
    input.addEventListener('change', this.processFile)
    input.innerHtml = "Open file";
    this.appendChild(input);
  }
}
customElements.define('custom-file', CustomFile);

```

---

<div class="post-metadata">

**Author:** ![system](https://canada1.discourse-cdn.com/flex035/uploads/elm_lang/original/1X/50a05e53677a2c3b47776d7abd0f113eb50193a1.png) [@system](https://discourse.elm-lang.org/u/system)\
**Post date:** [May 19, 2019, 2:04am UTC](https://discourse.elm-lang.org/t/bytes-ports-and-uint8array/3569/9 "2019-05-19T02:04:39Z")

</div>

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.