Simplest (and cheapest) way to upload/store/retrieve images

[badlydrawnrob]

I was a bit surprised to learn that Elm didn’t have a way to write images to some dedicated folder, but file upload seems reasonably easy (although I’m not sure how you’d handle errors).

What I’m looking for is anyone who’s successfully used native code to:

1. Upload an image,
2. Rename the file (which a lot of image upload sites do)^[1]
3. Post to the server with Http.post or Http.request,
4. The server renames the file to a random name and returns json,
5. Retrieve the URL of a successfully stored image (and any metadata),
6. Store the URL in the model, ready to add to a form.

So it’s a two-step form: Upload the image, send to the server. There are a few image servers that we could look at:

1. ImgBB (low-cost, limited customer service)
2. FreeImage.host (poor customer service, CORS errors)
3. Cloudinary (untested, has Ai tools)
4. Cloudflare Images (I can only find a handful of tutorials for API)
5. Scripta by @jxxcarlson (useful if you’re an admin uploading images)
6. Amazon S3 (I find Amazon not very user-friendly)

Right now ImageBB or CloudFront are possibly the cheapest, and you can start using them for free. I’m not sure how reliable ImageBB is, so I’d back up all your images. The alternative is to have a server-side image upload and use ports and flags, I guess.

It seems like quite an essential feature of any app to upload files, so I’m wondering how people are handling it currently. I’m not much a fan of ports, but if it’s simple to understand that’s another option.

I’ve seen a few posts on the topic, but some seem outdated or a bit complex.

---

The solution

Thanks to @wolfadex @passiomatic @jxxcarlson @Laurent

Dealing with images at such a low-level is new to me, so there was a lot of information to sift through, which I’ve written documentation about here (base64, multipart, cross-origin, etc). Depending on your perspective, this should be easier to do in Elm with packages or frameworks in future.^[2]

1. A working ImgBB implementation with Http.request and Http.stringPart
   • It uses MultiPart and the equivalent of Curl --form
   • Here’s how a successful response looks
2. A URL POST implementation that should work (but FreeImage.host has CORS errors) — any other image server that accepts image=<data-string> post should work.

Both these options should cover image servers, with variations.

Other notes

@wolfadex suggests to follow this components guide when implementing within a form. Also take care of naming your modules.

You’ll also need to wrap your head around Task which is something that can Never fail. You’ll probably want to upload multiple images so Task.sequence seems the way forward. As for UI you might like to track the progress of the upload.

All that is probably going to need a tutorial — there’s quite a lot of gaps in the documentation/examples for getting these things done. If you’re like me, you’d prefer higher-level detail than lower-level implementations. Hopefully there’ll be more off-the-shelf solutions in future.

Security concerns?

The only security concerns I can think of right now is exposing the API key. Uglifying your javascript may go some way to solving this, but I guess the better way would be storing it in backend code somehow.

---

1. Helps avoid duplicate names if the filename can be randomised ↩︎

2. Each server handles uploads slightly differently, but it’d be great for Elm Studio to handle all this complicated stuff for you in future. ↩︎

[passiomatic]

When Elm Designer was online it used Imgbb to store bitmap images (not SVG) uploaded by users of the app.

API access is free and after signup you can get a key to make calls. It is just a HTTP POST call. I consider the service quality at “demo level” but if you need to cook up something quickly I would use it.

You can grab the Imgbb module here elm-designer/src/Imgbb.elm at master · passiomatic/elm-designer · GitHub

You have to do some wiring, especially for the track upload progress which is important for the UX. Take a look here elm-designer/src/Main.elm at master · passiomatic/elm-designer · GitHub and subsequent Msg’s.

The Elm Designer wiring code is quite complex, because the app allows you to select multiple image(s) from disk or drag&drop them, and show process for each upload. You might not need all this complexity.

[badlydrawnrob]

Hey @passiomatic yes for now I think I prefer jQuery-style simplicity, without worrying about too many bells and whistles

So far I’ve made a few basic tests of another server (I’ve been a bit put off Imgbb due to negative reviews, but I guess you get what you pay for). It seems like you can upload local files with FILES["source"] but I can’t figure out how to do that with CURL.

As for your Imgbb module, it would be great to see some comments so I can follow it a bit better (there’s a lot of new stuff there, and I’m not sure what Http.track is doing — is that the track upload progress you’re talking about?)

Yeah your Msg has a LOT going on (with a lot of stuff I don’t understand)! From a UX perspective that’s probably nicer, but I’ve decided that to keep my learning light for now (I’m pretty much a one-man-band at the moment!), so a simple button suits me. Perhaps with a progress bar.

Fancy UI looks cool, but it seems to add a lot of overhead. I think I’m also going to have to look into breaking up a file and Msg into their own modules. There’s a lot to keep in your head there.

So TL;DR:

1. Is attaching a local file possible? Or must you convert it to base64?
2. Is toString the way to convert to base64?
3. https://freeimage.host/ automatically renames the file, so (2) is covered …
   • Just for posterity, how might you rename the file?
4. It seems a bit simpler now I know it’s just a Http.post
   • What’s the benefit of multipartBody over fileBody?
5. So [image/jpg] limits ONLY to files with jpg extension? (Seems to be)
6. Finally, in your Imgbb example I don’t see your API key
   • Are you providing this in config somehow for security?

Seems like I’m getting closer!

[wolfadex]

We use S3 and native browser/Elm file upload for handling PDFs at work. On the Elm side there hasn’t been too much too do, most of the work has had to been server side with scoping files between users and the actual handling of storing/retrieving files.

---

@jxxcarlson might have more thoughts on images specifically as I believe he’s done some work here with his Lamdera apps.

[badlydrawnrob]

Oh ok. Yes I definitely don’t want to deal with anything server side. I’d hire someone to do that I reckon. Any clues from @jxxcarlson would be much appreciated.

This is my current (mostly working) attempt — This is my current (mostly working) attempt

1. I keep getting a damn server Http.Error.
   • See this file for a working URL.
   • Funnily enough even with curl a base64 url string is throwing a server error^[1]
2. @wolfadex Yup I figured that out after double checking the csv file example.
   • File.toUrl converts an image to a base64 url string …
   • However, it seems .png files are NOT converting?

I’ve sent a support message to that image hosting site, perhaps I’ll try Imgbb as well, make sure it’s not an Elm problem. Perhaps it’s due to localhost not being https?

---

1. {"status_code":400,"error":{"message":"Invalid base64 string.","code":120},"status_txt":"Bad Request"} ↩︎

[wolfadex]

I think Elm doesn’t like modules and package name clashes (like File)

Somewhat correct. If you have 2 modules named Rob and try to

import Wolf

Elm doesn’t have a way to differentiate between them. You can, if you want, do

import Wolf
import Not.Wolf as Wolf

and then you can do Wolf.function. This can cause issues though if both modules expose functions/types with the same name.

What if our Task.perform fails? And what does Never mean. It can never fail?

Never is a value that can never happen. So yes Task.perform having an error value of Never means it can never fail.

but we still need to unpack the bloody Maybe

Why don’t you send the model.image along with in your message since at the point where the message is sent you know exactly that your image isn’t a Nothing?

Should File.name function be the second param to Task.perform?

Why do you send the file name in the message instead of setting it directly on the model?

Should File.toString be File.toUrl?

File.toString gives you the string contents, like for a text file, while File.toUrl gives you a base64 encoded string, excellent for use with <img />.

[badlydrawnrob]

Thanks @wolfadex I guess I’ll wrap my head around modules as I improve. It’s probably a better idea to make sure there’s no naming conflicts with elm/... packages. For now I’m using File_.SomeModule.

1. Thanks for clarifying that Never. It feels a bit weird to me!
2. I didn’t think to do that. Do you mean Just url → onClick (SendToServer url)?
3. Why do you send the file name in the message instead of setting it directly on the model?
   • In the csv example he’s using a Task to convert to a string …
   • So I thought to do the same thing with the File.toUrl …
   • At which point (in the update) you wouldn’t be able to grab the name anymore.
   • Is there a better way to do this?

I think the server problem is either me doing something wrong in the url, or the server has the problem, as I’ve encoded and decoded a few base64 images and they’re not posting with curl either.

Thanks!

[wolfadex]

@badlydrawnrob

It’s probably a better idea to make sure there’s no naming conflicts with elm/... packages. For now I’m using File_.SomeModule.

You should be fine with File.SomeModule. If anything I’d give a more accurate name instead of File. E.g. your modules seem to deal with images specifically so a more helpful name might be File.Image or ImageFile or even Image.

I also wouldn’t worry specifically about the elm/* packages. It’s more of a general issue, like if you used elm-ui 1.1.8 then you wouldn’t be able to name any of your local modules Element. I’d focus more on giving your modules names that make sense when you’re looking at the name along.

@badlydrawnrob

I didn’t think to do that. Do you mean Just url → onClick (SendToServer url)?

Exactly!

@badlydrawnrob

In the csv example he’s using a Task to convert to a string …

Which makes sense because a csv file is a specialized text file. If you open it in your text editor it’s still legible.

@badlydrawnrob

• At which point (in the update) you wouldn’t be able to grab the name anymore.
• Is there a better way to do this?

ImageSelected file ->
      ( model
      , Task.perform (ImageLoaded (File.name file)) (File.toString file)
      )

ImageLoaded filename content ->
      ( { model
            | image = Just content
            , imageName = filename
        }
      , Cmd.none
      )

can be converted to

ImageSelected file ->
      ( { model | imageName = File.name file }
      , Task.perform ImageLoaded (File.toString file)
      )

ImageLoaded content ->
      ( { model
            | image = Just content
        }
      , Cmd.none
      )

there’s no need to pass the name through the message+update loop again (as best I can tell).

---

A small stylistic opinion. I’d merge all of your File_.* modules into a single module. A module should generally be a container of sorts around a specific type. I’d also recommend Components | Elm Land for a very solid guide about how to model your component like modules.

[Laurent]

Cloudflare storage has a free tier: Pricing | Cloudflare R2 docs

[Laurent]

Also, to store thumbnails I resize them first on the client (with JS canvas).

[Laurent]

I just found out that Cloudflare images recently became available for free plans.

[jxxcarlson]

I’ll try to post something useful on this in the near future. I developed a user interface for scripta.io which offers a little database in which you can store images. The database, such as it is, is built using Cloudflare.

((Note: please view scripta in Firefox, Opera, or Safari. It is currently not displaying properly in Chrome (dunno why). Also, the UI is undergoing a major and much-needed facelift which is absorbing most of my energy these days, so there are various things that are broken. Working on it … ))

Below is a screenshot. You can (1) click on “Paste” to paste in an image or (2) click on “Upload from disk”. Either will load the image into a database attached to your scripta user account. To get a URL for your image for use in scripta or anywhere else, click on “Copy image address”.

Note that the image database is searchable (see the “filter” input field). You can tag images by clicking on “Edit.” If I put “bird” in the search box, all the images with a tag containing “bird” will be displayed. Put “blue bird” to retrieve images tagged by both blue and bird.

You can try this out either by creating an account on scripta or by signing into the “whatever” account with password hocuspocus716. I will leave that up until/if it is abused. Not by any of us, of course! You get to the image upload page by clicking on the camera icon in the vertical toolbar. See image below.

[jxxcarlson]

Screenshot of image database in the whatever account of scripta.io. It contains four items, but when filtered by “bird blue”, only one is displayed. The query “bir blu” accomplishes the same end.

[jxxcarlson]

I’ve been thinking for some time about offering a super low cost free-standing version of the above-mentioned image library, but haven’t freed up the time yet to do that yet. It will be an all-Elm solution to the problem. In the meantime, if you would like to, you can sign up for a scripta account and use this feature. I plan for scripta to be around for a long time (e.g., longer than me, ha ha!)

[badlydrawnrob]

Sorry for the late response all, I’ve been recovering from the dreaded covid.

@wolfadex Yes for a proper app your suggestion to be more specific makes sense. This particular directory is sort of a “how do I do ___ in Elm?” folder, so I’ve grouped them by elm/package or feature. I think I’m getting the hang of naming modules, and where to take care.

Thanks for clearing up the File.name for me — of course that’s the way to do it! I guess I’m not entirely clear of when and why to use Task.perform (it could be as simple as looking at the type signature of a function like File.toString. My guess is that for multiple files, you’d use Task.sequence?

Finally, on your point about splitting/consolidating modules — I was following @passiomatic how he did things, but I’ll take a look at the article you’ve mentioned. Is it a matter of taste?

---

@Laurent It seems the Cloudflare images would be the better option, I imagine it’s a little simpler to handle. Hopefully the pricing won’t get out of hand. Would you say it’s better to have a dedicated image server?

---

@jxxcarlson A walkthrough tutorial would be much appreciated. I have the basics down (some servers don’t accept cross-domain POST which is frustrating) but for the fancy UI stuff, I’m not sure I’d want to dive into that without a bit of hand-holding.

I’m not sure how Evan’s Elm Studio is going to look, but I feel Elm should be heading towards simplicity for the everyday feature set. Images is an integral part of apps (as you’ve mentioned, ideally it’d be browser-agnostic) and not everyone wants to dive into low-level detail (like base64).

Your images app looks very cool and I definitely think there’s space for a low-cost image server (not just for Elm apps). The key for me would be well-documented API, customer service response (or a great FAQ), cross-origin, programmatic resizing etc, so on. I think a randomised image name is preferable, with the metadata available, and likely a thumbnail (custom crop of original image) — and the ability to limit filesize (or have visitors resize it on the fly). As it stands I run the risk of visitors having a poor user-experience as I don’t expect them to know what “compress” or “filesize” really means

Is it demo-ready, or production-ready as-is?

[badlydrawnrob]

@jxxcarlson One possible bug I’m seeing is adding a tag to an image duplicates the image?

[jxxcarlson]

Thanks, you are correct! I’ll get it fixed later this weekend.

PS. Merry forthcoming Christmas!

[Laurent]

@badlydrawnrob

Would you say it’s better to have a dedicated image server?

Well it depends if you want to switch career from dev to devops