Upload files to S3

Show and Tell
1

I release a package to upload files to AWS S3!

https://package.elm-lang.org/packages/jaredramirez/elm-s3/latest/

10 Likes
2

Will need that … tomorrow :slight_smile:

3

Just a heads up from a security perspective that one should be careful exposing an AWS accessKey and secretKey in frontend code. There is a more secure way around this which is using an S3 Presigned Url. The caveat of that is it requires getting the presigned url to the browser in the first place.

If interested I wrote an explanation how to do that towards the end of this post (see Uploading videos from the web). Here’s a bit of Elm code anyone should feel free to borrow as well -> https://github.com/ckoster22/stabilizer

10 Likes
4

Please please having credentials in your code which by definition are dowloaded to all your clients, and they can use that at will is a real security issue. So at minimum should be explicitly mentioned.

2 Likes
5

@charliek @berend

Thank you both for pointing that’s out. I’ve updated the README to directly warn users about the security risk. Thanks for the link to using presigned urls, I’ll definitely look into how they work and see if they can be used to this package to do things more securely!

2 Likes
6

Great! Just scrolling down, progress reports would be very useful for any upload taking over 2 seconds I would say. Normally http progress reports are pretty easy with Elm, perhaps restructure your code?

7

I agree that it would be helpful, the problem is that uploading to S3 (at least from my understanding here), requires the current time. That’s why it uses Time.now with tasks, but at the moment you can’t track http tasks (github issue) so the user would have to pass in the current time which isn’t ideal.

Regardless it’s probably a good thing to include the option to track, even if it’s not the ideal experience!

8

I re-released with extra information on the possible security risks. Thanks again for bringing this up, this is definitely something essential for anyone who uses this package to know.

Here’s the snippit added:

Note on security (Please read)

This package requires storing your AWS secret and access keys in your Elm code. Please note that there is a security concern with having copies of these keys downloaded to each client that uses your app.

Probably the best way to do S3 uploads is to store your AWS keys safely on your server, when your user is authenticated generate presigned URLs, send that back to the client, the upload to that URL.

That being said, this can be done less securely using only the browser. Because this package uses AWS secret and access keys, the best way to mitigate any risks is to make sure that your IAM user’s policy only allows for PutObject and PutObjectAcl and only has access to the subdirectory within your bucket that want you upload files to.

My user’s IAM policy looks like this:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "s3:PutObject",
                "s3:PutObjectAcl"
            ],
            "Resource": [
                "arn:aws:s3:::my-bucket/uploads/*" // SPECIFY YOUR SUB DIRECTORY HERE
            ]
        }
    ]
}

If your keys fell into the wrong hands with this policy, the most access they’d have is the ability to upload and overwrite any files already uploaded. If you add unique IDs and timestamps to each filename that you upload, the possiblility of a file being overwritten become unlikely.

Please takes this information into consideration before using this library and proceed with caution.

9

:+1: @jaredramirez

Regarding tracking progress, that is something being done in the code I linked to earlier. Was there something different between this snippet of code and other ways of tracking progress? Http.request allows tracking progress.

10

See my comment above about tracking: Upload files to S3

Basically, the http package doesn’t allow tracking on the request task and we chain Time.now with the http task to get the current time and use that to form the request.

11

Pre-signed URL is a great way to provide an upload link to third parties, since S3 API already knows how to take care of very big files etc.

Additionally, I think this S3 package will work well with Elm running on AWS using:

https://package.elm-lang.org/packages/the-sett/elm-serverless/latest/

Run that way, since the code is not available to the client you do not have the same issue with exposing the access keys.

12

Alternatively one can generate temporary access keys which are scoped to whatever resource(s) you want clients to access using AWS’s auth service, Cognito (e.g as in this guide) which can then be safely passed to the client.

1 Like
13

I did not know about that.

Is it usual to directly open up AWS services to clients using this? I am just wondering as I am working on getting the GitHub - ktonon/elm-aws-generate: Generate an Elm interface to an AWS service stuff working on 0.19, so that AWS services can be used from within elm-serverless.

They can be used more directly from within the client too, using a temporary access token? Seems to open up a lot of possibilities, just not sure if its a good idea or not. Perhaps we should have an Elm+AWS thread some time.

1 Like
14

I’ve dug a little bit through Elm’s Kernel Code this weekend, and this seems to be a limitation / implementation detail of how the platform currently works.

Essentially, to send messages back to the update function, you need access to a Platform.Router value to pass to Platform.sendToApp. This value gets passed into the onEffects “update” function of effect managers. Now, the way to think about Cmd and Sub values in this context is “A Task bound to an effect manager” - which means when using Http.request, the request gets run by the Http effect manager, which can construct the underlying Task in a way such that the progress event handler has access to that Platform.Router, and can therefore send progress messages back to us.

When using Http.task >> Task.attempt instead, the Http module has to construct the task immediately without a Platform.Router, and that Task then gets run by the Task effect manager, which is more generic and has no notion of progress at all.

1 Like
15

Thanks for looking into this! Probably the best way to allow tracking is to expose another function that uses Http.request instead of Http.task and takes the config/file as well as the current time. It’s not the ideal developer experience, but would allow tracking for those who need it.

16

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.