# XML endpoints with the-sett/elm-aws-core

**URL:** <https://discourse.elm-lang.org/t/xml-endpoints-with-the-sett-elm-aws-core/5293>\
**Category:** Learn\
**Created:** [March 10, 2020, 10:23am UTC](https://discourse.elm-lang.org/t/xml-endpoints-with-the-sett-elm-aws-core/5293 "2020-03-10T10:23:02Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![Ulrar](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.elm-lang.org/ulrar/32/3025_2.png) [@Ulrar](https://discourse.elm-lang.org/u/Ulrar)\
**Post date:** [March 10, 2020, 10:23am UTC](https://discourse.elm-lang.org/t/xml-endpoints-with-the-sett-elm-aws-core/5293/1 "2020-03-10T10:23:02Z")

</div>

Hi,

I’m using @rupert the-sett/elm-aws-core to get information from the AWS API, which unfortunately is very inconsistent. Most of the endpoints return JSON and that works fine with that lib (I’m using auto scaling and ecs without issues), but the EC2 endpoint returns XML.

The lib doesn’t have any options not to decode JSON as far as I can tell, which does not work at all :

```elm
let ec2 region = Service.defineRegional "ec2" "2016-11-15" Service.QUERY Service.SignV4 (Service.setXmlNamespace "https://ec2.amazonaws.com/doc/2016-11-15/") region in
let params = [("Action", "DescribeImages"), ("Version", "2016-11-15"), ("Owner.1", "self")] in
Http.request "DescribeImages" GET "/" Http.emptyBody JSONDECODERHERE |> Http.addQuery params |> Http.send (ec2 region) creds |> Task.attempt msg

```

```auto
Failed : Problem with the given value:

"<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<DescribeImagesResponse .......

```

As you can see in there you need to pass a JSON Decoder to Http.request, but that, of course, fails when receiving XML. I asked the question on SO where it was suggested to use elm’s HTTP package directly, which can return the raw string instead of trying to parse it, but that doesn’t look simple to integrate with the package, or at least I don’t see how without re-implementing a good chunk of it.

Any suggestions on how to query the EC2 API ?

Thank you

---

<div class="post-metadata">

**Author:** ![rupert](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.elm-lang.org/rupert/32/1775_2.png) [@rupert](https://discourse.elm-lang.org/u/rupert)\
**Post date:** [March 10, 2020, 11:23am UTC](https://discourse.elm-lang.org/t/xml-endpoints-with-the-sett-elm-aws-core/5293/2 "2020-03-10T11:23:40Z")

</div>

Hi,

AWS services cover 5 possible Protocols, which are described by the `Protocol` enum, which I’ve annotated with some comments to describe what each is like:

```elm
type Protocol
    = EC2 -- XML based, requests always POST, endpoint name in header.
    | JSON -- JSON based, requests always POST, endpoint name in header.
    | QUERY -- Query param based.
    | REST_JSON -- JSON REST API
    | REST_XML -- XML REST API

```

Unfortunately, I have only got as far as implementing and testing support for the `JSON` protocol.

It looks like you are wanting to use EC2, which is the odd one out; as far as I can tell, only the EC2 service uses this protocol.

The intention is to add support for all AWS services, but its a big task and I am not committed to a timeline for completing it. However, good to know that there is some demand, and I will record this in an issue on the GitHub repository at least in order to guide prioritisation.

#### Things you could do to get it working.

1. Perhaps you could use the `JSON` protocol, but just use `Decode.string` to get the whole response body as a String? and then do the XML parsing yourself on that String. Not 100% sure this will work, its a hack but might get you up and running if it does work.

2. You can do the HTTP request using the `elm/http` package easily enough. The tricky part is the V4 signing of the request. You can find the code for that below, either try to use it as-is, or copy paste into your own project and adapt it as needed there:

> <https://github.com/the-sett/elm-aws-core/blob/master/src/AWS/Core/Signers/V4.elm#L157>

1. Fork `elm-aws-core` and add support for EC2 yourself. Will be happy to support getting a good patch out of that and incorporating it into the published package too.

---

<div class="post-metadata">

**Author:** ![ymtszw](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.elm-lang.org/ymtszw/32/2360_2.png) [@ymtszw](https://discourse.elm-lang.org/u/ymtszw)\
**Post date:** [March 10, 2020, 11:36am UTC](https://discourse.elm-lang.org/t/xml-endpoints-with-the-sett-elm-aws-core/5293/3 "2020-03-10T11:36:10Z")

</div>

Hi @Ulrar!

If you are going to take the pass #1 @rupert suggested, and only interested in “decodeing” XML responses (not “sending” XML document), please take a look at [elm-xml-decode](https://package.elm-lang.org/packages/ymtszw/elm-xml-decode/latest/) which I maintain.  
It is much like `Json.Decode`, with some twists for XML characteristics. If you are already familiar with `Json.Decode`, it should not be a big gap.

The workflow shoule be like this: you accept XML responses from APIs as raw strings using `Json.Decode.string`, then plug the result into `Xml.Decode.run` with your own `Xml.Decode.Decoder YourData`.  
Contact me wherever (GitHub issue could be nicer), if you find any difficulties. :chears:

---

<div class="post-metadata">

**Author:** ![Ulrar](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.elm-lang.org/ulrar/32/3025_2.png) [@Ulrar](https://discourse.elm-lang.org/u/Ulrar)\
**Post date:** [March 10, 2020, 11:49am UTC](https://discourse.elm-lang.org/t/xml-endpoints-with-the-sett-elm-aws-core/5293/4 "2020-03-10T11:49:07Z")

</div>

Hi,

I did try #1 yesterday already actually, hoping to use elm-xml-decode (I already wrote the decoder with it, looks very easy indeed !) and unfortunately that’s what generates the error above. I assume Json.Decode.string still tries to decode the whole thing as json, it’s probably expecting the whole thing to be wrapped in curly brackets.

I’ll try to get that working then, and if I do get it working I’ll make sure to make a PR with my code.  
Thanks !

---

<div class="post-metadata">

**Author:** ![rupert](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.elm-lang.org/rupert/32/1775_2.png) [@rupert](https://discourse.elm-lang.org/u/rupert)\
**Post date:** [March 10, 2020, 2:52pm UTC](https://discourse.elm-lang.org/t/xml-endpoints-with-the-sett-elm-aws-core/5293/5 "2020-03-10T14:52:07Z")

</div>

Just having a think about it… on the request you build (which is confusingly called `Unsigned`), a `Decoder` needs to be included:

```auto
type alias Unsigned a =
    { name : String
    , method : String
    , path : String
    , body : Body
    , decoder : Decoder a
    , headers : List ( String, String )
    , query : List ( String, String )
    , responseParser : Maybe (Http.Response String -> Result Http.Error a)
    }

```

If I were to change the type of that `decoder` field to `String -> a` instead of `Decoder` it will keep things more open, and you would be able to pass in your own `String -> a` function that decodes the String into XML into whatever Elm records you like.

Actually I think the decoder needs its type to be `String -> Result String a`, to be able to handle the case where decoding fails (with an error message).

It will be possible to do this as `decoder` is only used in the context of `Decode.decodeString` for decoding string responses as JSON.

---

<div class="post-metadata">

**Author:** ![rupert](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.elm-lang.org/rupert/32/1775_2.png) [@rupert](https://discourse.elm-lang.org/u/rupert)\
**Post date:** [March 10, 2020, 3:00pm UTC](https://discourse.elm-lang.org/t/xml-endpoints-with-the-sett-elm-aws-core/5293/6 "2020-03-10T15:00:53Z")

</div>

Also, I forgot about this feature that is already there:

[https://package.elm-lang.org/packages/the-sett/elm-aws-core/latest/AWS-Core-Http#setResponseParser](https://package.elm-lang.org/packages/the-sett/elm-aws-core/latest/AWS-Core-Http#setResponseParser)

This lets you override the `Decoder` and instead provide your own handler for the entire HTTP response. You should be able to get the response body from an `Http.GoodStatus_` response and decode to XML that way?

---

<div class="post-metadata">

**Author:** ![rupert](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.elm-lang.org/rupert/32/1775_2.png) [@rupert](https://discourse.elm-lang.org/u/rupert)\
**Post date:** [March 10, 2020, 3:33pm UTC](https://discourse.elm-lang.org/t/xml-endpoints-with-the-sett-elm-aws-core/5293/7 "2020-03-10T15:33:43Z")

</div>

Ok, I published a new version. The old `Http.request` function is renamed to `Http.requestWithJsonDecoder`, and has been replaced by one that takes the decoder as a `String -> Result String a`.

[https://package.elm-lang.org/packages/the-sett/elm-aws-core/4.0.0/AWS-Core-Http#request](https://package.elm-lang.org/packages/the-sett/elm-aws-core/4.0.0/AWS-Core-Http#request)

> <https://github.com/the-sett/elm-aws-core/issues/1>
>
> https://discourse.elm-lang.org/t/xml-endpoints-with-the-sett-elm-aws-core/5293
> Ulrar wants to use XML with the EC2 service. Right now a JSON decoder is required, which makes thi awkward.
> Change the...

Hopefully, this will get you over the line?

---

<div class="post-metadata">

**Author:** ![Ulrar](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.elm-lang.org/ulrar/32/3025_2.png) [@Ulrar](https://discourse.elm-lang.org/u/Ulrar)\
**Post date:** [March 10, 2020, 4:22pm UTC](https://discourse.elm-lang.org/t/xml-endpoints-with-the-sett-elm-aws-core/5293/8 "2020-03-10T16:22:23Z")

</div>

That works perfectly ! I was trying to get that to work with setResponseParser when you published that, but with the new Http.request I’m able to use elm-xml-decode and parse the XML.  
Now I’ll try to get the Batch endpoint to work, which works yet another way of course.

Thanks a lot for the help and quick new version !

---

<div class="post-metadata">

**Author:** ![ymtszw](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.elm-lang.org/ymtszw/32/2360_2.png) [@ymtszw](https://discourse.elm-lang.org/u/ymtszw)\
**Post date:** [March 10, 2020, 4:42pm UTC](https://discourse.elm-lang.org/t/xml-endpoints-with-the-sett-elm-aws-core/5293/9 "2020-03-10T16:42:45Z")

</div>

Great agility @rupert!

---

<div class="post-metadata">

**Author:** ![rupert](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.elm-lang.org/rupert/32/1775_2.png) [@rupert](https://discourse.elm-lang.org/u/rupert)\
**Post date:** [March 13, 2020, 3:58pm UTC](https://discourse.elm-lang.org/t/xml-endpoints-with-the-sett-elm-aws-core/5293/10 "2020-03-13T15:58:26Z")

</div>

> [@rupert](#):
>
> AWS services cover 5 possible Protocols, which are described by the `Protocol` enum, which I’ve annotated with some comments to describe what each is like:
> 
> ```auto
> type Protocol
> = EC2 -- XML based, requests always POST, endpoint name in header.
> | JSON -- JSON based, requests always POST, endpoint name in header.
> | QUERY -- Query param based.
> | REST_JSON -- JSON REST API
> | REST_XML -- XML REST API
> 
> ```
> 
> Unfortunately, I have only got as far as implementing and testing support for the `JSON` protocol.

I misrepresented where things are at with this explanation.

In addition to `elm-aws-core` I am working on [Elm AWS Codegen](https://github.com/rupertlssmith/elm-aws-codegen). This aims to generate service stubs in Elm for all AWS services. It will effectively give you one function for each endpoint, and a data model descibing all the requests and responses so you never have to get down to building these all yourself. It is the code generator that only supports the JSON protocol so far.

`elm-aws-core` should probably be able to support them all, but I have not yet tested it on all.

I’ve added an issue on the code generator for tracking services there is a demand for:

> <https://github.com/the-sett/elm-aws-codegen/issues/1>
>
> Was asked about the iot service on slack. That's a REST-JSON one.

---

<div class="post-metadata">

**Author:** ![system](https://canada1.discourse-cdn.com/flex035/uploads/elm_lang/original/1X/50a05e53677a2c3b47776d7abd0f113eb50193a1.png) [@system](https://discourse.elm-lang.org/u/system)\
**Post date:** [March 23, 2020, 4:09pm UTC](https://discourse.elm-lang.org/t/xml-endpoints-with-the-sett-elm-aws-core/5293/11 "2020-03-23T16:09:34Z")

</div>

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.