Prototyping a simple JSON API: best practices and gotchas

Learn
1

Perhaps this could be better asked as a task:
:star: You’re designing a Json/REST API for an intermediate student of Elm.
:student: What would it look like? What would you include? What wouldn’t you?
      What materials would you use to teach it?

TL;DR: I need to understand API design on a higher level, but don’t want to go too far into the weeds, as I won’t be the one building out the architecture eventually. A lot of amateur/intermediate developers could use a solid guide to get started with Elm and json, heavily leaning towards simple documentation/guides.

API specs like JSON:API are complicated and overkill for this students needs, so would be great to get feedback on where to look for lighter, changeable, APIs. Understand this might be a bit too wide-ranging as questions go, but for students at this level it’s hard to know where to start.

Case Study: A Bad API?

I’ve been experimenting with the Open Library API which you can see in action here on Ellie App. I have to say, I don’t like it. I’m new to API design and doing some skim-reading about it, but their API seems inconsistent, messy, and not particularly pleasurable to work with:

  1. Search and Work/Editions APIs seem to have no order:

    • IDs are given rather than Book objects
    • IDs are in any order, in any language as far as I can tell, which makes the search not very useful for English readers

  2. Chained Http.get requests are needed for fields like "authors" (line 160)

    • The thread link above looks useful, but I only half understand it
    • It seems a lot of effort to build up a proper Book type in this way.

  3. “Orphaned” structures like "authors": [{"key": "/authors/OL34184A"}]

    • Nearly all the books I’ve searched for only have this single object inside a list. It seems a pointless complication?
    • Or it’s premature optimisation or forward planning?

  4. Lists like "isbn_13": ["9780140328721"] where it’s mostly a single entry (why not just use a String?

  5. Having to build URLs for "covers": [8739161] with the covers api, rather than supplying a full url I can pull in.

  6. Having to use Json.Decode.maybe A LOT instead of Json.Decode.nullable, because they’re missing (and not null)

  7. The API redirects book ISBNs to their book IDs (luckily Elm allows for redirects).

You can see my full comments (and gripes) in the Ellie App comments.

I’m all for simplicity

Caveats that are important to me, personally:

  • I’m not a serious programmer and will hire eventually.
  • I prefer simplicity wherever possible, avoiding complexity in the API.
  • It’s likely to be mostly static, without a backend, using JsonBin or Postman.
  • POST, GET, and PUT are needed, and data normalisation may be required.
  • It will be changeable (possibly often), but shouldn’t forward-plan too much.
  • By the end you should be able to explain high-level concepts to another colleague or employee.

The API

  • Which books would you suggest reading (ideally quickly)?
  • Which videos are good to watch?
  • How do you spot a BAD Api?

Elm Lang Specific

  • Should we prefer a flatter API? as Elm prefers it’s Model?
  • Should we prefer simpler data structures?
    • i.e: avoiding cases like "authors" above?
    • It’s possible to turn complicated structures to simpler ones, with Json.Decode.map, but it adds overhead.
  • What are the Gotchas that come up again and again?

Some Answers

2

half this seems Elm related the other half seems Rest api related. perhaps better understanding REST will answer some. but one thing id like to note is, you’re not beholden to their api shape.

Lists like "isbn_13": ["9780140328721"] where it’s mostly a single entry (why not just use a String?

what I mean is in your model you don’t need to hold a List String
you could decode it as a single integer.
Decode.field "isbn_13" (Decode.index 0 Decode.int) (something like that)
and on your Model have an isbn : Int field.

the idea being you can transform the REST api data into whatever you want through decoders.
only decode the data you need and into the shape you need it in.

Having to build URLs for "covers": [8739161] with the covers api, rather than supplying a full url I can pull in.

that’s another api request to get the covers. chaining api reqs with Task.andThen is pretty common.

Having to use Json.Decode.maybe A LOT instead of Json.Decode.nullable, because they’re missing (and not null)

Decode.oneOf is sometimes nicer. you can try a decoder that has the required field, and if it doesn’t, then fall through to another decoder.

or you can make them optional and provide a default.

* It seems a lot of effort to build up a proper `Book` type in this way.

yep. that’s why tools like graphql exist. but idk if this book api supports it.

1 Like
3

@m0n01d Right. I did say it was wide-ranging! Perhaps a better question would be:

:star: If you could design the perfect Elm Json API, what would that look like?
:student: (especially if you had to teach it to an intermediate student of Elm)

But that’s too broad and varied a question. I’m green on RESTful systems and prefer my prototyping to be as simple as possible, so I do think it’s worth thinking about the interplay between Elm decoders and Json structures: I guess I’m asking for pointers/resources on all the above.

Shape

  • Good call on shape, I didn’t know Decode.index existed! That’s handy.
  • Some more examples changing shape would be appreciated if they’re out there.

Building a type (from chained requests)

Task.andThen in the example I linked to confuses me, and articles like this one make my brain fall over. Do you store intermediate values in the model? Then build your Book once all requests are complete?

These kind of situations call for a detailed walkthrough course, in my opinion. They seem advanced to me.

GraphQL

I’ve dabbled in the past. Do you think it beats Json APIs in most cases?

4

elm/json is really weird: It’s missing a function for decoding optional fields. (Fields that are there only sometimes.) Instead, it has the Decode.maybe kludge, which does this: “If my decoder fails for whatever reason, return Nothing instead”. It’s meant to catch “this field does not exist” errors, but it also catches all other reasons a decoder can fail, masking mistakes you make in your decoder, or that the JSON producer makes.

It’s better to use Json.Decode.Extra.optionalField.

2 Likes
5

Yeah I’d agree, and I’m also leaning towards your suggestion of being explicit with nullable rather than missing values. I also know there’s Json.Decode.Pipeline.optional and they had quite a bit of discussion about how to deal with optional fields. In fact, the tables on that link is handy to see how a function deals with the json and would be a nice edition to the package docs.

6

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.